PROBLEM: Import process appears to be working, but ends abruptly (usually at the same point in the overall process on every occasion). e.g. Attempting to import 100 records always stops at record 38.
SOLUTION: More often than not, these situations are the result of improperly formatted data. Some common issues that cause these type of problems include:
- Errant commas within the data
- Carriage return values within the data
- Field mismatches between records
Below is an example of a set of records that demonstrates proper formatting:
Row 1:first,last,phone_home,email
Row 2:angel,magana,888.555.1212,angel@example.com
In the above example, Row 1 represents the column headers or labels, while Row 2 represents an actual record. Were one to assume that each value in the rows is separated by a comma, inspection of Row 1 and Row 2 would reveal that there is an equal number of values in Row 1 as there are in Row 2. That's exactly what we want. Any type of difference would cause a problem.
The folowing demonstrates a file that would cause a problem:
Row 1:first,last,phone_home,email
Row 2:angel,magana,888.555.1212,angel@example.com
Row 3:john,doe,888.555.1313,john@example.com,123 Main Street
Other common problems can be addressed via PHP.INI modifications and are related to:
- Low max_execution_time, max_input_time, post_max_size or upload_max_filesize values
The first two values control the amount of time PHP will allow to expire before it stops processing a script. A setting of 300 should be more than sufficient for either.
The last two settings are used to control the amount of data that can be accepted. A value of 40M or higher should suffice.
Please visit the official PHP site for further details.
What about the "description" column that often spans many lines with unwanted characters?
ReplyDelete@Jonny
ReplyDeleteThat falls into the category of "carriage return values within the data."
Attempting to import multi-line values from a CSV source only leads to frustration and little progress.
If you must import free form text data, you'll need to use a different source format. I usually leave the data in its SQL state and use other tools to import for that type of scenario.
Some popular tools are SnapLogic, Apatar and Talend.