Min 1 Minute interval between scans if data source
Hi @Jon .
We’ve done a load of testing the last week. The CLI does actually work with AE open.
We have found an issue though, or at least what we consider an issue. After doing a ton of working, it seems that even on the CLI there is a large delay of about 20 seconds when getting JSON data from a URI. It seems the bot just hangs for no reason, since the data is there and it shouldn’t take longer than milliseconds to get it.
Can this be fixed?
@Jon After doing some further testing, it seems to be a bug, as this behaviour does not happen when you run the render with all the same settings (including the remote JSON source) from the GUI Templater. This only occurs when running Templater from the CLI.
@Jon After doing some further testing, it seems to be a bug, as this behaviour does not happen when you run the render with all the same settings (including the remote JSON source) from the GUI Templater. This only occurs when running Templater from the CLI.
EDIT: It definitely seems to be a bug. When replicating instead of rendering, pulling in remote JSON data takes less than a second. Only we rendering it takes up to 45 seconds to pull in the same data.
@Singit this is a known issue with the CLI and JSON polling intervals. We have addressed it, fixed it, and it is in the 2.9 release which we’ll announce tomorrow — Feb. 25, 2019. In that release, with the CLI, Bot’s polling rate can low as 0.5 minutes in the 2.9 release. Thank you for your patience as we make our product more robust.
@ariestav We actually aren’t using the ‘bot’ functionality. We are doing a simple ‘render’ through the CLI, so there should only be one instant poll. This works when you use the ‘replicate’ function, but not when you use the ‘render’ function.
@Singit Thank you for the clarification. Maybe AE is taking its time to prepare for actual rendering — loading output modules, render settings, etc.? Are you certain that the issue is related to data retrieval? We have noted the issue and will try to reproduce it.
I have seen cases where the CLI can take much longer to complete when rendering due to Templater making use of AE’s render engine. When you launch AE’s with its GUI normally, it’s likely that its rendering engine modules are loaded automatically, but via the cli it can feel like it’s the cause of delay. With replication, there is no use of AE’s rendering engine — the layers are invalidated with new data and then the project is copied to a new file, so it is speedier in execution.
What do the time stamps in templater.log file say?
@ariestav I’ve been pretty careful to find where the bug lies. I’ve tested it out pretty much every which way possible from both the GUI and CLI. I’ve also checked the task manager to see the resource usage. It really just hangs doing nothing from 30-60 seconds. This is also shown in the log file. I just updated and it also still occurs in the updated version. We are running it with the GUI open so that it doesn’t need to load AE. But I did test without the GUI as well and it is the same.
Here is a tidied up and redacted log file…
2019-2-23 @ 15:26:44 - Session started
2019-2-23 @ 15:26:44 - Templater is searching for manufacturer's licensing framework.
2019-2-23 @ 15:26:44 - Initializing Templater CLI
2019-2-23 @ 15:26:44 - Starting Command Line Interface with options file.
2019-2-23 @ 15:26:44 - Main options
log_location => C:\xxxxx
data_source => https://xxxxx
aep => C:\xxxxx
source_footage => C:\xxxxx
output_location => Y:\xxxxx
render_settings => Yap
output_module => Yap
output_prefix =>
row_start => 1
row_end => 5
save_on_completion => false
quit_on_completion => false
quit_on_errors =>
tasks => [object Object]
prefs => [object Object]
2019-2-23 @ 15:26:44 - Prefs options
default_target => Icon_1
tsv_from_excel => false
replica_comps_to_q => false
replica_comps_to_a => false
remove_replicas_am => false
fix_layer_exp => false
missing_footage => 3
placeholder_visual => C:\xxxxxx
placeholder_audio => C:\xxxxxx
unique_downloads => false
suppress_ae_dialog => true
silence_logging => false
purge_caches => false
remote_json => [object Object]
bot => [object Object]
2019-2-23 @ 15:26:44 - Bot options
name => kaori
goog_freq => 5
json_freq => 1
max_queued => 5
action_render => true
action_replicate => false
pre_cmd_data =>
pre_jsx_data =>
pre_cmd_batch =>
pre_jsx_batch =>
pre_cmd_job =>
pre_jsx_job =>
pre_cmd_update =>
pre_jsx_update =>
pre_cmd_output =>
pre_jsx_output =>
post_cmd_data =>
post_jsx_data =>
post_cmd_batch =>
post_jsx_batch =>
post_cmd_job_async => true
post_cmd_job => C:\xxxxxxx
post_jsx_job =>
post_cmd_update =>
post_jsx_update =>
post_cmd_output =>
post_jsx_output =>
enable_cmd =>
enable_jsx =>
shutdown_cmd =>
shutdown_jsx =>
2019-2-23 @ 15:26:44 - Remote JSON options
username =>
password =>
headers => [object Object]
2019-2-23 @ 15:26:44 - Remote JSON headers
2019-2-23 @ 15:26:44 - Task options
render => true
replicate => false
bot_enable => false
2019-2-23 @ 15:26:44 - WARNINGS regarding CLI options
2019-2-23 @ 15:26:44 - There is no specified output prefix. Consider adding an output prefix to group batch output together
2019-2-23 @ 15:26:44 - Examining URI
https://xxxxxx
2019-2-23 @ 15:26:44 - Detected URL as a remote JSON resource
https://xxxxx
2019-2-23 @ 15:26:46 - Configuring Templater's preferences to match values in templater-options.json file
2019-2-23 @ 15:26:46 - Setting Templater's preferences to match values in options file
2019-2-23 @ 15:26:46 - Executing tasks with the following settings
2019-2-23 @ 15:26:47 - Templater is executing requested tasks as specified in options
2019-2-23 @ 15:26:47 - Executing batch render of video output
2019-2-23 @ 15:26:47 - Data source is
Remote JSON
2019-2-23 @ 15:26:47- Using start row
1
2019-2-23 @ 15:26:47 - Using end row
5
2019-2-23 @ 15:26:47 - Retrieving objects 1 through 5 from from remote JSON feed. Please wait...
2019-2-23 @ 15:27:26 - Now rendering target for rows 1 through 5
2019-2-23 @ 15:27:26 - Render job data
2019-2-23 @ 15:27:26 - Performing layout for 6 layers
2019-2-23 @ 15:27:26 - [ --__== DEPTH 0 ==__-- ]
2019-2-23 @ 15:27:29 - ---- RENDERING TARGET ----
2019-2-23 @ 15:27:43 - ---- DONE RENDERING TARGET ----
2019-2-23 @ 15:27:43 - Asynchronous command executing on [ bot_post_job ] event
2019-2-23 @ 15:27:43 - Completed rendering rows 1 through 5.
Many thanks for these details. They will help us troubleshoot. It does seem strange that retrieval begins at 12:26:47, but then the next message that reads Now rendering target for rows 1 through 5 is stamped 39 seconds after that @ 15:27:26. We will try to reproduce — are you using any kind of authentication scheme for the remote JSON feed?
Thanks,
Arie
@ariestav Yeh, it’s strange. The file being rendered is also very very simple. It normally takes about 1 second to render when done manually from the GUI.
We aren’t using any authentication on the remote JSON feed.
@ariestav Any update on the bug?
EDIT: It’s now becoming a bigger and bigger issue. It now hangs for about 2 minutes when pulling in data from the JSON source. It is getting to the point where we are looking at no longer using templater bot, as we haven’t had any indication this will be fixed.