0

We have a custom order export script that runs out of memory. Exporting a small time range works fine, but exporting all of December does not (more orders than usual, due to the holidays.)

I added some log statements to give me the memory usage at various points in the process, and ran the export for a small time range. It appears that the spike happens when rendering the report.

Beginning of controller action Memory: 5.84M <database query happens at this point> after getReport() Memory: 19.48M <now Magento prepares our HTML output> End of controller action Memory: 44.14M *When exporting a large dataset, "End of controller action" never gets reached.

We really only need to use the CSV export feature, but we can't even get to that because you have to "Refresh" the grid before clicking "Export."

Short of re-writing this as a stand-alone report (outside of Magento) I'm not sure what to do. Anyone?

  • How much available memory your PHP has? – Kirby Jan 8 '15 at 1:16
  • By the way, what kind of entities you going to export? – Kirby Jan 8 '15 at 1:43
  • It is an export of order items. Columns include: order_id, order_status, product_id, product_sku, product_quantity, product_price, customer_email, customer_name – Eric Seastrand Jan 8 '15 at 15:09
  • Looks like you have very limited memory at your disposal, which is kind of a problem if you intend to manipulate collections (or to run a Magento in general) – Julien Lachal Jan 8 '15 at 17:06
  • server memory limit is 512mb. – Eric Seastrand Jan 8 '15 at 19:45
1

So what I ended up doing was hacking the report so that you can run the CSV export without loading the grid first, by specifying ?as_csv at the end of the URL.

First, I added this little snippet to the indexAction to effectively re-route the request to the exportCsvAction if ?as_csv is set in the URL.

public function indexAction() {
    if(isset($_GET['as_csv'])) {
        $this->exportCsvAction();
        return;
    }

    $this->_initAction()
            ->renderLayout();
}

Now we can take any URL generated by the date picker, add ?as_csv to it, and get a CSV export for that date range.

Then I added this Javascript code to the report index page (via xml layout):

// This is how the "Refresh" button generates the URL for the date range.
varienGrid.prototype.getFilter = function(){
    var filters = $$('#'+this.containerId+' .filter input', '#'+this.containerId+' .filter select');
    var elements = [];
    for(var i in filters){
        if(filters[i].value && filters[i].value.length) elements.push(filters[i]);
    }
    if (!this.doFilterCallback || (this.doFilterCallback && this.doFilterCallback())) {
        return this.addVarToUrl(this.filterVar, encode_base64(Form.serializeElements(elements)));
    }
}
// This is the function that the "Export" button calls. Overwrite it and add the ?as_csv queryString
varienGrid.prototype.doExport = function() {
    window.location = this.getFilter() + '?as_csv';
}

While this isn't the most elegant solution, is is a reusable one for anyone else with this unique issue.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.