I received a question about the performance of eConnect when importing large GL Journal Entries, such as a JE with 1,000 lines.
In some prior eConnect / GP load testing, I had only imported JEs with up to 500 lines, so I didn't know the answer.
I just fired up my Batch Load Test import tool and tested some eConnect imports of GL JEs with lots of lines.
Here's what I found. Your import times may vary depending on your environment, but this is some baseline data to consider.
Importing a single JE with these line counts took:
500 lines: 2 seconds
1,000 lines: 4 seconds
2,000 lines: 8 seconds
4,000 lines: 16 seconds
8,000 lines: 31 seconds
10,000 lines: 39 seconds
15,000 lines: 54 seconds
20,000 lines: 77 seconds
So it looks like eConnect can pretty easily import a standard JE with quite a few lines without any issues.
CAN you import a JE with 20,000 lines? Yes.
SHOULD you import a JE with 20,000 lines? I would not recommend it as a routine process. Break up the data into more manageable JE sizes. If nothing else, it will make reconciliations and research more manageable.
Note that this is a standard GL JE without Analytical Accounting. In my experience, eConnect performance for a JE with AA is horrible. Definitely keep your JEs small and your batches small when importing with AA data.
Steve Endow is a Microsoft MVP in
Los Angeles. He is the owner of Precipio Services, which provides
Dynamics GP integrations, customizations, and automation solutions.
Hello Steve, I've come across your blog and it has been super helpful for me as I have ventured into trying to use eConnect for data integrations with GP. After reading an older post on using an array of multiple transactions and the speed you received from that, I decided to give it a go. We have a monthly process where we need to push close to 50,000 sop invoice line items to GP 2015 R2. I've set up a .NET application using an array for multiple transactions and the memory stream to create the xml document. I have tested this with 475 line items and I am getting a time of about 32 seconds. Does that seem right? Does the type of transactions lend to the time variance? SOP Transaction vs JE Transaction?
ReplyDeleteKaren
Hi Karen,
DeleteI have not yet tried importing SOP transactions with lots of lines, so I don't currently have a benchmark for that.
In my SOP Invoice import testing, I imported transactions with 4 lines each, and they imported at a rate of about 7 transactions per second. Unfortunately that probably isn't a helpful benchmark for you.
SOP Invoices will definitely import at a different rate than JE, as eConnect is doing a lot more work in the background for SOP.
OK. Thank you for getting back to me!
ReplyDeleteDid you observe any performance improvement when you submitted multiple SOP transactions to eConnect per request?
DeleteIf you aren't seeing any improvement or are getting unexpected results, let me know and I can try a test with 500+ line items per invoice.
I have been submitting the 475 line transactions (with the 11 header transactions) in one eConnect request and getting that 32 second performance. Maybe that is par for the course. I'm going to try upping that to a few thousand and see what I get. As you mentioned, perhaps there is just a lot more work being done in the background for SOPs.
ReplyDeleteJust to confirm, you are importing 11 SOP Invoices with a total of 475 lines (average of 43 lines?), in a single eConnect request, and that is taking 32 seconds?
DeleteOr are you importing 11 SOP Invoices with 475 lines in each invoice?
First scenario...11 SOP Invoices with total of 475 lines.
ReplyDeleteI've modified my eConnect load test app to compare import of individual transactions vs multiple transactions ("single send").
DeleteTesting now...
These times will probably vary significantly based on several factors but here are some samples.
ReplyDeleteImporting 50 invoices with 50 line items each:
Individual send: 50 SOP Invoices imported in 59 seconds
Single send: 50 SOP Invoices imported in 34 seconds
42% improvement
100 invoices with 50 lines each:
100 SOP Invoices imported in 91 seconds
100 SOP Invoices imported in 69 seconds
24% improvement
100 invoices with 100 lines each:
Individual send: 100 SOP Invoices imported in 157 seconds
Single send: 100 SOP Invoices imported in 132 seconds
16% improvement
I've written a new blog post with the SOP Invoice import benchmarks:
Deletehttps://blog.steveendow.com/2020/04/how-long-does-it-take-to-import.html
Thank you, Steve! I will look at this tomorrow. Appreciate your time and attention very much.
Delete