503 Service Unavailable error during data import

[BR]

I am trying to import Sales Orders and Line items form a CSV file of about 28,000 records. If I attempt to import all of them, iDempiere crashes, and I get a "503 Service Unavailable" message, and the only way I know how to fix that problem is by rebooting the server.

Here is what I have tried so far:

NOTE: For each failure, some of the records succeeded, but since I don't have a good way of telling which failed and which succeeded, I deleted all imported records before the next attempt.

1. Import all 28,000 records together. That lead to a 503.
   
2. I then broke out a portion of the records into 6 chunks of ~100 records each. The first 5 worked, but I got the 503 error on the 6th.
   
3. I imported the 6th chunk by itself, and it worked.

Based on #3, it doesn't seem like a problem with the data itself.

Anyone have an idea about what is happening here, or how I can get more information about where the problem may be?

Thanks,

Brandon

[redhuan d. oon]

Can you do a test on GardenWorld with scrubbed data : Client details changed to Joe Block etc. and product keys to GW's. Just to eliminate whether it is systemic or your client data mapping issue. If it still fails then its systemic, send me that test data (scrubbed)  for me to test and figure out.

[Carlos Antonio Ruiz Gomez]

Hi Brandon, the CSV Import is not intended to manage big amounts of data.

We have discussed that in past and I think most people explore which is the limit on their installation (I guess mostly memory dependent) and then split the big files in corresponding pieces.

Regards,

Carlos Ruiz


El 13/09/16 a las 00:13, BR escribió:

[BR]

Importing with scrubbed data as you suggested appears to work. Any idea about how I can figure out what the problem with my data might be? Is there a log that is written to before iDempiere crashes that I can look at for clues?

[BR]

I noticed that the crashes correlated with hitting 100% cpu utilization on the AWS instance I was running iDempiere on. After increasing the instance size to include more resources, this worked as expected for all 28,000 records.

Thanks