I am working on a module that requires access to some tabular data provided from an external source. It's only two columns but has roughly 40000 rows.
Currently, my module is just parsing the CSV in entirety when necessary. This works well, but since the file is about 450Kb, this will eat up server resources once deployed to a production site.
I'd like to move this data to a Magento table and am having trouble.
I'm using the RDBMS methods in my setup script, ala:
$installer = $this; $installer->startSetup(); $table = $installer->getConnection() ->newTable($installer->getTable('my_table')) ->addColumn('column_a', Varien_Db_Ddl_Table::TYPE_TEXT, 5, array( 'nullable' => false, ), 'Column A') ->addColumn('column_b', Varien_Db_Ddl_Table::TYPE_TEXT, 3, array( 'nullable' => false, ), 'Column B'); $installer->getConnection()->createTable($table); $installer->endSetup();
This works just fine to create my table, but actually getting my data in is wracking my head (I'm still a noob when it comes to database management).
Ideally, I would just parse my CSV file and insert values into the table without having to deal with a bunch of copy/paste.
Are there any built-in methods for handling arbitrary CSV data, or are they all built to handle just what they need to? What would be the best approach to get my data into Magento?