Hi

Whenever I invoke my robot passing a different input var I get this error very often:

Error from the Load Page action. Could not load from the URL "http://mydomain.mysite/mypage". Too many open files
Load Page - For Each Tag[1] - Extract URL - Load Page
{.[0]..}


Robot fetches a different part of a specific site for each var input. It does this by looping through 50 urls from an index page, and fetching content on each of the pages.

Is it possible that your server keeps every file open for too long after each run of a robot? Is there a limit of files a robot can open on each run, that I should keep in mind?

thanks

Maybe "OpenKaput" would be a better name for the service... at least while it's in beta.

Yeah... kinda looks like it. Too unreliable.

Anyway... I reduced that loop from 50 to 15 links but the "too many open files" error still comes up way too often, together with this one: "The client-side distribution policy was unable to find an available RoboServer".

I guess it's back to PHP DOM scraping for me. At least for the time being.

 

PS: And to think I actually even installed XP on my mac for this. :(
 

When it works, it works well. When it works. Which is not very often, lately.

 Color me 'dissatisfied.'

Hey all.  Some legitimate complaints.  Unfortunately, due to the openness of the system, some people are flooding the servers which creates these issues.  We are in the process of building some system monitoring tools to help address this.  Stay tuned.

We very much appreciate being kept in the loop on your challenges. It helps us decide whether to stick it out and keep using your excellent tools while you solve the issues, or migrate elsewhere because of the uptime problems.

All,

  We performed some significant system maintenance last night.  We've doubled the memory on the main server.  For those who care about details, the database tracking statistics had grown too large and was causing corruption and performance issues.   This has been fixed.

  Stability will be significantly improved, but please let us know if you see any issues.  Thanks for your patience.