Mirror
in package
implements
CrawlConstants
This class is responsible for syncing crawl archives between machines using the SeekQuarry/Yioop search engine
Mirror periodically queries the queue server asking for a list of files that have changed in its parent since the last sync time. It then proceeds to download them.
Tags
Interfaces, Classes, Traits and Enums
- CrawlConstants
- Shared constants and enums used by components that are involved in the crawling process
Table of Contents
- DOWNLOAD_RANGE = 50000000
- Maximum number of bytes from a file to download in one go
- $db : object
- Reference to a database object. Used since has directory manipulation functions
- $last_notify : string
- Last time the machine being mirrored was notified Mirror.php is still running
- $last_sync : string
- Last time a sync list was obtained from master machines
- $last_sync_file : string
- File name where last sync time is written
- $name_server : string
- Url or IP address of the name_server to get sites to crawl from
- $parent_url : string
- Url of the Yioop instance we are mirroring
- $start_sync : string
- Time of start of current sync
- $sync_dir : string
- Directory to sync
- $sync_schedule : string
- Files to download for current sync
- __construct() : mixed
- Sets up the field variables so that syncing can begin
- checkScheduler() : mixed
- Gets status and, if done processing all other mirroring activities, gets a new list of files that have changed since the last synchronization from the web app of the machine we are mirroring with.
- copyNextSyncFile() : mixed
- Downloads the next file from the schedule of files to download received from the web app.
- loop() : mixed
- Main loop for the mirror script.
- start() : mixed
- This is the function that should be called to get the mirror to start syncing. Calls init to handle the command line arguments then enters the syncer's main loop
Constants
DOWNLOAD_RANGE
Maximum number of bytes from a file to download in one go
public
mixed
DOWNLOAD_RANGE
= 50000000
Properties
$db
Reference to a database object. Used since has directory manipulation functions
public
object
$db
$last_notify
Last time the machine being mirrored was notified Mirror.php is still running
public
string
$last_notify
$last_sync
Last time a sync list was obtained from master machines
public
string
$last_sync
$last_sync_file
File name where last sync time is written
public
string
$last_sync_file
$name_server
Url or IP address of the name_server to get sites to crawl from
public
string
$name_server
$parent_url
Url of the Yioop instance we are mirroring
public
string
$parent_url
$start_sync
Time of start of current sync
public
string
$start_sync
$sync_dir
Directory to sync
public
string
$sync_dir
$sync_schedule
Files to download for current sync
public
string
$sync_schedule
Methods
__construct()
Sets up the field variables so that syncing can begin
public
__construct(string $name_server) : mixed
Parameters
- $name_server : string
-
URL or IP address of the name server
Return values
mixed —checkScheduler()
Gets status and, if done processing all other mirroring activities, gets a new list of files that have changed since the last synchronization from the web app of the machine we are mirroring with.
public
checkScheduler() : mixed
Return values
mixed —array or bool. Returns false if weren't successful in contacting web app, otherwise, returns an array with a status and potentially a list of files ot sync
copyNextSyncFile()
Downloads the next file from the schedule of files to download received from the web app.
public
copyNextSyncFile() : mixed
Return values
mixed —loop()
Main loop for the mirror script.
public
loop() : mixed
Return values
mixed —start()
This is the function that should be called to get the mirror to start syncing. Calls init to handle the command line arguments then enters the syncer's main loop
public
start() : mixed