Wait for list to load? One of my lists takes 45seconds to load 14290 items. Please find compositions attached.
Data Set Load Time.zip (326 KB)
After checking this out more, this is indeed the same issues as: https://community.vuo.org/t/-/5326
However my data is quite huge (2MB) so maybe the fix wouldn’t be the same? Would the new “Structured Data” node set be more suited to this job? (Or a custom data loader which only outputs specific chunks of CSV data?)
I have done some testing, and although it doesn’t have to do with this node, I remade a text file reader node that automatically takes the data and chops it up using strtok(). I have attached the node & data. This allows data to be read with no noticeable lag at all. I created a CSV that was 17 MB and 114,320 and it loaded as the composition ran.
The issue with ‘split text’ must be huge because Vuo is so fast!
If anyone want to use the node its attached. I needed to make this to read in data values from a 3 axis sensor placed on an actor. Each time I was working on the project it would take 5 mins to load the data. Now instant. Any issues source is attached.
split_text_file.zip (3.83 KB)
@alexmitchellmus nice job !
Ran some tests, about 1 minute between the two. Blue is the built-in, magenta yours.
Test made with a 130 k CSV file of 6000 values
@jstrecker, is it ok to use strtok() in a node - or should I change it too strtok_r()? I thought vuo keeps nodes executing on each thread, only multi threading different nodes?
So it needs to be strtok_r() otherwise more than 2 nodes corrupts the other ones list! :-(
Fix attached. Try to do this with strtok()! :-D
split_text_file_reentrant.zip (3.95 KB)
After checking this out more, this is indeed the same issues as: Split Text node becomes slow with long text
Confirmed. Since the Split Text bug is already fixed in our dev code, we tested your composition and CSV with it.
Hey @jstrecker is the new code in vuo as fast as the example I posted using strtok_r? I really do need fast data reading! :-) 1.3 looks like its developing nicely! Great Job can’t wait!!!
The new code parses your CSV file instantly (human perception; haven’t timed it).
For Load structured data (XML, JSON, CSV, TXT) from files or network (HTTP, HTTPS) we’ll be adding a node for parsing CSV that supports double-quote escaping, which Split Text does not.
Absolutely Fantastic! :-)
