Thursday, November 29, 2018

How long does it take to import a Dynamics GP GL Journal Entry with LOTS of lines?

By Steve Endow

I received a question about the performance of eConnect when importing large GL Journal Entries, such as a JE with 1,000 lines.

In some prior eConnect / GP load testing, I had only imported JEs with up to 500 lines, so I didn't know the answer.

I just fired up my Batch Load Test import tool and tested some eConnect imports of GL JEs with lots of lines.

Here's what I found.  Your import times may vary depending on your environment, but this is some baseline data to consider.

Importing a single JE with these line counts took:

500 lines:  2 seconds
1,000 lines:  4 seconds
2,000 lines:  8 seconds
4,000 lines:  16 seconds
8,000 lines:  31 seconds
10,000 lines:  39 seconds
15,000 lines:  54 seconds
20,000 lines:  77 seconds



So it looks like eConnect can pretty easily import a standard JE with quite a few lines without any issues.

CAN you import a JE with 20,000 lines?  Yes.

SHOULD you import a JE with 20,000 lines?  I would not recommend it as a routine process.  Break up the data into more manageable JE sizes.  If nothing else, it will make reconciliations and research more manageable.


Note that this is a standard GL JE without Analytical Accounting.  In my experience, eConnect performance for a JE with AA is horrible.  Definitely keep your JEs small and your batches small when importing with AA data.


Steve Endow is a Microsoft MVP in Los Angeles.  He is the owner of Precipio Services, which provides Dynamics GP integrations, customizations, and automation solutions.

You can also find him on Twitter and YouTube




11 comments:

  1. Hello Steve, I've come across your blog and it has been super helpful for me as I have ventured into trying to use eConnect for data integrations with GP. After reading an older post on using an array of multiple transactions and the speed you received from that, I decided to give it a go. We have a monthly process where we need to push close to 50,000 sop invoice line items to GP 2015 R2. I've set up a .NET application using an array for multiple transactions and the memory stream to create the xml document. I have tested this with 475 line items and I am getting a time of about 32 seconds. Does that seem right? Does the type of transactions lend to the time variance? SOP Transaction vs JE Transaction?

    Karen

    ReplyDelete
    Replies
    1. Hi Karen,

      I have not yet tried importing SOP transactions with lots of lines, so I don't currently have a benchmark for that.

      In my SOP Invoice import testing, I imported transactions with 4 lines each, and they imported at a rate of about 7 transactions per second. Unfortunately that probably isn't a helpful benchmark for you.

      SOP Invoices will definitely import at a different rate than JE, as eConnect is doing a lot more work in the background for SOP.

      Delete
  2. OK. Thank you for getting back to me!

    ReplyDelete
    Replies
    1. Did you observe any performance improvement when you submitted multiple SOP transactions to eConnect per request?

      If you aren't seeing any improvement or are getting unexpected results, let me know and I can try a test with 500+ line items per invoice.

      Delete
  3. I have been submitting the 475 line transactions (with the 11 header transactions) in one eConnect request and getting that 32 second performance. Maybe that is par for the course. I'm going to try upping that to a few thousand and see what I get. As you mentioned, perhaps there is just a lot more work being done in the background for SOPs.

    ReplyDelete
    Replies
    1. Just to confirm, you are importing 11 SOP Invoices with a total of 475 lines (average of 43 lines?), in a single eConnect request, and that is taking 32 seconds?

      Or are you importing 11 SOP Invoices with 475 lines in each invoice?

      Delete
  4. First scenario...11 SOP Invoices with total of 475 lines.

    ReplyDelete
    Replies
    1. I've modified my eConnect load test app to compare import of individual transactions vs multiple transactions ("single send").

      Testing now...

      Delete
  5. These times will probably vary significantly based on several factors but here are some samples.


    Importing 50 invoices with 50 line items each:

    Individual send: 50 SOP Invoices imported in 59 seconds
    Single send: 50 SOP Invoices imported in 34 seconds
    42% improvement


    100 invoices with 50 lines each:

    100 SOP Invoices imported in 91 seconds
    100 SOP Invoices imported in 69 seconds
    24% improvement


    100 invoices with 100 lines each:

    Individual send: 100 SOP Invoices imported in 157 seconds
    Single send: 100 SOP Invoices imported in 132 seconds
    16% improvement

    ReplyDelete
    Replies
    1. I've written a new blog post with the SOP Invoice import benchmarks:

      https://blog.steveendow.com/2020/04/how-long-does-it-take-to-import.html

      Delete
    2. Thank you, Steve! I will look at this tomorrow. Appreciate your time and attention very much.

      Delete

All comments must be reviewed and approved before being published. Your comment will not appear immediately.

How many digits can a Business Central Amount field actually support?

 by Steve Endow (If anyone has a technical explanation for the discrepancy between the Docs and the BC behavior, let me know!) On Sunday nig...