<![CDATA[Data Source size limit?]]>Hi,

We’re testing out using local JSON as a data source for our projects. In Google Sheets, data is composed of about 150 columns, and can range anywhere from 1 to 750 records. I’ve done a test with about 30 records which took a little while for the JSON to load (the file is loaded from the local hard drive that has the .aep) which rendered much more quickly than a Google Sheet due to minimal delay between renders. I then ran a test with a file of about 500 records which never actually loaded, it took so long. Would you expect this to be the case? Is there a maximum number of key:value pairs and jobs that we should consider? Thanks.

Pete

]]>https://forums.dataclay.com/topic/408/data-source-size-limitRSS for NodeFri, 10 Apr 2026 21:12:46 GMTThu, 09 Apr 2026 18:59:56 GMT60<![CDATA[Reply to Data Source size limit? on Fri, 10 Apr 2026 16:56:42 GMT]]>@ariestav @Jeff Thanks for the info. sounds like we’d be better off just using Que?!

]]>https://forums.dataclay.com/post/1279https://forums.dataclay.com/post/1279Fri, 10 Apr 2026 16:56:42 GMT<![CDATA[Reply to Data Source size limit? on Fri, 10 Apr 2026 15:20:26 GMT]]>@pbretz Adding onto what @Jeff wrote, if you are going to have a single entry in the local JSON file, you’d want to use a render-status property key in the JSON object so that Bot would know when the over-written file was ready to be processed.

]]>https://forums.dataclay.com/post/1278https://forums.dataclay.com/post/1278Fri, 10 Apr 2026 15:20:26 GMT<![CDATA[Reply to Data Source size limit? on Fri, 10 Apr 2026 14:15:29 GMT]]>@pbretz

The main issue with this scenario is the amount of memory After Effects allocates to ExtendScripts like Templater. Adobe doesn’t want third-party apps to have free rein over all a system’s memory, which means that loading a large Data Source, as you’ve described here, can be a bit unwieldy. The problem is that Templater has to open and read the entire dataset with these limited resources, which is causing a significant slowdown.

To get around this, we’d need to feed Templater the data in smaller chunks. The easiest way to achieve this would probably be to use a JSON file with only one entry and then use an Event Script on the “After Output” Templater Event to update the JSON with the next record. That would probably be the most efficient method and should keep the time required to parse the JSON to a minimum.

Another option would be to use a custom JSON URL feed to pass one record at a time to Templater for processing. This method also requires an Event Script to update the JSON record after each render, since Templater can’t write back to the URL feed to update the render-status. However, this can also be a very efficient way to pass data to Templater, as you can customize the URL feed internally.

Hopefully, that all makes sense, but if you have any other questions, let us know.

Thanks,

Jeff

]]>https://forums.dataclay.com/post/1277https://forums.dataclay.com/post/1277Fri, 10 Apr 2026 14:15:29 GMT