Php tiempo de ejecución para siempre y analizadores web

so i found a nice slice of code, well its best to say a theoretical example of what might be the code for a decent web parse. I recall that when i got into this issue myself a while back i could not get the script to run for an infinite time span in fact i couldn't have it run more then a few hours.

This is after i set the following:

    set_time_limit(0);
    ini_set("memory_limit", "800M");
// in some case i would set them both in others, just one or the other

I have been doing some reading as to how to get PHP to run for a longer time span, an infinity stretch to be exact. I found many suggestions for cron job instead of PHP yet i would like to find a way to have this done in php.

I would love some examples, even theoretical if thats all you can muster.

I would like to use this block of code i mentioned above, which i found here as a reference to get the discussion started on the right path.

¿Cómo escribir un rastreador?

  while(list of unvisited URLs is not empty) {
     take URL from list
     fetch content
     record whatever it is you want to about the content
     if content is HTML {
     parse out URLs from links
     foreach URL {
        if it matches your rules
          and it's not already in either the visited or unvisited list
          add it to the unvisited list
     }
   }
 }

preguntado el 02 de febrero de 12 a las 10:02

I found many suggestions for cron job instead of PHP yet i would like to find a way to have this done in php. - these are not mutually exclusive. You can easily set up a cron job to run a PHP script. For something like this, it would be what you want to do - you don't want to site with a web browser open on a page waiting for it to complete the task. When you run PHP from the command line (which you are doing if it is set up as a cron job) you don't need to worry about the time limit, at the command line there is no limit by default. -

@DaveRandom A for insistence would be lovely. -

What's the problem? We're running php script for weeks in work and it just goes fine. -

Entonces, ¿cuál es exactamente tu pregunta? -

@Vyktor say this parser was running with two arrays set to function as memory, do you think they would run all the same ? -

1 Respuestas

Utilizan cronjobs WITH php not instead of.

You can run PHP scripts as a cron on a Linux server as follows:

<time/frequency> <path to PHP> <php script full path>

e.g. This will run every minute...

* * * * * /usr/bin/php -q /var/www/html/cron/parser.php

Or on Windows you can use the Task Scheduler to run the script, which is located in Control Panel, on Windows 7 it is in Administrative Tools within Control Panel.

I have used a combination of PHP, MySQL, Curl and crons to have a web application run indefinitely until it's parsed all the data I want to strip from URLs.

Respondido 02 Feb 12, 16:02

How about running it on windows ? is that possible ? - RmH

I'm running PHP via wamp, is it all the same? - RmH

Thanks buddy, one last thing though, any alternatives ? i am having - RmH

For Windows, none that I am aware of, Windows Task Scheduler is pretty simple to set up. - Leo Haris

No es la respuesta que estás buscando? Examinar otras preguntas etiquetadas or haz tu propia pregunta.