The file you are looking for it here:
vendor/magento/module-catalog-widget/view/frontend/templates/product/widget/content/grid.phtml
If you place some dummy text here, you will see if on your home page. So if you want to edit this file further, you can copy it over to your theme.
To answer your question, at this point I have not found a .phtml or .xml that controls the product grids, they are all controlled by the styles based on css breakpoints. If you take a look at this file vendor/magento/theme-frontend-blank/Magento_Catalog/web/css/source/module/_listings.less
you will see this:
.media-width(@extremum, @break) when (@extremum = 'min') and (@break = @screen__m) {
.page-products .products-grid .product-item { width: 100%/3 }
.page-products.page-layout-1column .products-grid .product-item { width: 100%/4 }
.page-products.page-layout-3columns .products-grid .product-item { width: 100%/2 }
}
.media-width(@extremum, @break) when (@extremum = 'min') and (@break = @screen__l) {
.products-grid .product-item { width: 100%/5 }
.page-layout-1column .products-grid .product-item { width: 100%/6 }
.page-layout-3columns .products-grid .product-item { width: 100%/4 }
.page-products .products-grid .product-items { margin: 0; }
.page-products .products-grid .product-item {
width: 23.233%;
margin-left: calc(~"(100% - 4 * 23.233%) / 3");
padding: 0;
&:nth-child(4n+1) {
margin-left: 0;
}
}
.page-products.page-layout-1column .products-grid .product-item { width: 100%/5 }
.page-products.page-layout-3columns .products-grid .product-item { width: 100%/4 }
}
which defines the breakpoints at when the grid will shift from 5 to 4 to 3. When I have had to change this around, I just do it in the styles, based on what I need, just change the math around to work with the design.