X-Scripts

READY-MADE SOLUTIONS FOR YOUR BUSINESS

OUR CONTACTS:
Skype: igor_sev2
Email : order@x-scripts.com

Script to add ads on the website localmart.ru

In this article we will look at an example of writing a script to add ads on the website localmart.ru, real Estate section.
to
let's Start with the preliminaries - check the site and find out what fields are required and which we would like to fill later.

Before writing the script necessary to define the functional requirements script:

* Publish ads in one section: "Property => rent => Apartments, rooms"

* City: Moscow

* the Script should work constantly and post new ads as they appear in the specified directory.

* Authorization localmart.ru: no need, because we will write the contact information in the ad description.

* Ads will be taken from a text file with simple formatting rules.

* Text file in cat-th will be the announcement must be called "ad.txt".

* Ads are not ready for the publication (drafts) must be located in directories with a name starting with "[draft]".

* Posted: after posting the announcement directory announcement will be renamed:
  at the beginning of the name of the ads will be added "[published]", before publishing it:

  "Rent a room" after posting would be: "[published] Rent a room"

* After the publication of the advertisement in the file url.txt the link is backed up to the published announcement.

Description of the file format "ad.txt" with the content ads.

ad.txt is a plain text file with encoding "windows-1251". File format - ini (the same format is, for example, the configuration file PHP -- php.ini).
of
the Ini format is easy to read/edit the person, and also,with this format easy to work with from PHP.

All lines beginning with a semicolon ";" - comments and ignored by the script.
of
All fields listed in ad.txt -- mandatory.A fully working example of populating a ad.txt can be found in "data/ads/[draft] New ad/".

Determine the functionality you can start the algorithm of the script.

1) since the script needs to work constantly and as new ads to publish them, the script will run in an infinite loop.

2) to avoid leaks RAM and hangs HumanEmulator implement restart script every N iterations.

3) If you have your listing, pick the first available, if not, then wait N minutes and check again.

4) because after the anonymous ad localmart.ru automatically authorizes the user before each visit we will clean the browser.

5) Go to the page of the announcement.

6) Fill in the required fields.

7) uploads.

8) Published the announcement.

9) Get a reference to the published announcement and save it in the file url.txt

10) Rename the directory with the ad - added "[published]".

11) So as too not flood the Board after the publication of the advertisement must be adhered to pause before publication of the next announcement.

you can Now the implementation of the publisher.

Create a main script file "add_to_localmart.php" with basic content:

// coding: windows-1251

// Setup HumanEmulator
// -----------------------------------------------

// Running XHE
$xhe_host = "127.0.0.1:7010";

// HumanEmulator lib
require "../../Templates/xweb_human_emulator.php";

require "tools/functions.php";

// The script settings
// -----------------------------------------------

// The script
//-----------------------------------------------

get the script to post advertisements as they appear will make an infinite loop.

while(true)
{
//
} // end while true

To avoid leaks RAM and hangs HumaEmulator add to the script the ability to restart every N iterations.

the settings of the script will specify how many iterations to restart the script:

$restart = 10;

Add to the script the iteration counter (before the beginning of the while loop):

$iter_count = 1;

while(true)
{
//
}

you can Now add the implementation of the restart (at the beginning of the while loop):

while(true)
{
if($iter_count >= $restart)
{
$app->restart($debug->get_cur_script_path(), "", $app->get_port());
}
}

Next start receiving ads for publishing.

To keep the ads will be in the directory "data/ads". Looks kinda like this structure:


- data/

- ads/ : the list of declarations

- "Ready for publication of the announcement (name not important)/"

- img/ : photos for ads (photo by M. b. any)

- ad.txt : contents of announcements

- "[draft] the Draft announcements (will not be published)/"

- "[published] Published the announcement/"

- img/ : photos published to ad

- ad.txt : the content of the published ads

- url.txt : a link to the published announcement

this will add in settings the directory where will store the announcement and the pause time, if ad no:

// Where are the ads
$ads_dir = "data/ads";

// Waiting time (min.)
$wtm = 10;

In the implementation of the receipt of the advertisement for publication, you must note that ads may not be, and the ads are unable to get:

// Get the ad for publication
$ad = get_ad($ads_dir);

if($ad === false)
{ // Something went wrong
$app->quit();
}
elseif(!$ad)
{ // No ads
echo "Waiting for {$wtm} min." . PHP_EOL;

$iter_count++;

sleep(mins2secs($wtm));

continue;
}

echo "Published an announcement: '{$ad['title']}'" . PHP_EOL;

Implementation of get_ad() is quite simple:

* Choose one ad (ready for publication)

* Check if a file exists ad.txt

* Read and split it with POM. built-in PHPthe function "parse_ini_file"

* If there is a photo, stored full paths.

Before you go to the site it is necessary to clean browser (cookies, etc.) to reset the old authorization.

// Clean and customizable browser
$browser->close_all_tabs();
$browser->navigate("about:blank");
$browser->clear_address_bar_history();
$browser->clear_cache();
$browser->clear_history();
$browser->clear_cookies("", true, true);
$browser->recreate();
$app->clear();

if(!$browser->is_enable_java_script()) $browser->enable_java_script(true, false);

Now you can open the page of the announcement.

$browser->navigate("http://localmart.ru/add");

Start filling out the fields in the form of publication of the advertisement:

fill_fields($ad);

Because a lot of fields, we bring the implementation to a function.
The implementation itself filling in the fields is simple - enter the text in the desired field or click on list and select the desired item:

// Header
$input->set_focus_by_attribute("name", "Ad[title]");
$input->set_value_by_attribute("name", "Ad[title]", True, $ad["title"]);
sleep(1);

// Category: real estate
$anchor->click_by_inner_text("Select", true);
sleep(1);
$input->click_by_number_by_form_name(1, "item-add-form");
$input->set_value_by_number_by_form_name(1, "property", "item-add-form");
sleep(1);
$anchor->click_by_inner_text("Property");
sleep(1);

The pause we set, to have time to work on javascript.
Because, when you publish ads localmart.ru automatically creates an account for the specified email, the normal address will be mentioned in the description,
and in the spec. the field will generate a fake email.

For this we use the built-in ability HumanEmulator on the generation of the first and last name:

$submitter->generate_random_name(),
$submitter->generate_random_second_name()

you can Also randomize the fake domain email - we will draw up a simple list of domains for fake address and when generating'll choose a random domain.

$hosts = ["mail.ru", "rambler.ru", "gmail.com", "hotmail.com", "yandex.ru", "ya.ru", "yandex.com"];
$email = $submitter->generate_random_name()
. $submitter->generate_random_second_name()
. "@"
. $hosts[rand(0, count($hosts)-1)];
$input->set_focus_by_attribute("id", "Ad_email");
$input->set_value_by_attribute("id", "Ad_email", True, $email);

the Next step is to download the photos, if any.

if($ad["img"])
{
// ...
}

If there are photos, $ad["img"] will contain a list of strings with full paths to all the photos to the ad, if there is no photo, it will be just an empty array.

the photo uploading takes place in 5 stages:

* Copy photos to a temporary directory. Because Internet Explorer locks the directory from which it loads the files, it will not give us the opportunity to rename the directory with the announcement.

date_default_timezone_set("Europe/Moscow"); // Need to generate $p2
$img = [];
foreach($ad["img"] as $p)
{
// Get the file extension ".jpg", ".png"
$p_ext = explode(".", basename($p));
$p_ext = $p_ext[count($p_ext)-1];
// Generate a new path to a temporary file
$p2 = "res/tmp/" . (date_timestamp_get(date_create()) + rand(99, 9999)) . ".{$p_ext}";

copy($p, $p2);

$img[] = realpath($p2);
}

* Generated a string with the full path to the photo:

// Generated list of photos to download
$files = "";
foreach($img as $p)
{
if(!file_exists($p)) continue;

$files .= "\"{$p}\" ";
}

* Method to call $window->execute_open_file to HumanEmulator was expecting a dialog box to download the file.

$window->execute_open_file("Select output file",$files,"&Open", true, true);
* Click on the button that opens the dialog box.

* Delete temporary files. Better temporary files to delete after the restart HumanEmulator. For example, add this code before the while loop().

// Remove temporary files
if(!file_exists($tmp_path)) mkdir($tmp_path, 0777, true);
$tmp_path_list = scandir($tmp_path);
foreach($tmp_path_list as $p)
{
if($p === "." or $p === "..") continue;

unlink($p);
}

will Add the setting:

// Temporary files
$tmp_path = "res/tmp";

After the upload, you can click on the button to publish the announcement on the website:

$mouse->send_click($anchor->get_x_by_inner_text("advertise") + 10,
$anchor->get_y_by_inner_text("advertise") + 10);

Verify - whether a published ad?

$url2 = $webpage->get_location_url();
if($url2 !== "http://localmart.ru/features")
{
echo "[ERROR] Error publishing the ads." . PHP_EOL;

$app->quit();
}

Sometimes ads before you publish, can expect check,check actively whether our ad?

To do this, go to the page with active ad and look for it:

$browser->navigate("http://localmart.ru/cabinet/items?view=active");

$ad_link = $anchor->get_attribute_by_inner_text($ad["title"], true, "href");

if(!$ad_link)
{
// listing not active
}
else
{
// ad is active
}

If everything is OK, we save the reference to the Declaration in the file url.txt

file_put_contents("{$ad['_base_path']}\\url.txt", $ad_link);

And rename the directory to "[published] Ad name".

$new_path = str_replace($ad["_dir_name"], "[published] {$ad['_dir_name']}", $ad["_base_path"]);
rename($ad["_base_path"], $new_path);

Before going on the trail. iteration is necessary to increase the iteration counter "$iter_count"

$iter_count++;

Don't forget when you create new ads to name add the directory "[draft]" to the script missed the draft ads.

just keep in mind that if the advertisement is ready for publication, then you need before you publish:

* Close the text editor by file ad.txt

* Close promoterwise/photo editors with photos for ads.

* File Manager (Explorer, PC): to exit the directory with the announcement.

This is necessary in order for the script to rename the directory with the announcement.

* Remove from the name of the directory "[draft]".

this to the script I saw that the announcement can be published.


The script is written 24.04.2015 in Human Emulator 4.9.18 Advanced.

download the script

<< Other scripts