Integrating Yioop Libraries with Composer

Yioop provides a large variety of search-engine related and natural language related libraries that you might want to use in your own custom web application. Composer allows you to easily combine packages you need from several PHP projects into your project and manage their dependencies. In this article, we explore how to add Yioop to a project managed by Composer.

The Composer site describes how to download and install composer so that it can be run from the command line via commands like:

 composer some_directive_for_composer

Once you've installed composer you are set to create a new PHP project with it. Make a new folder, and open a command/terminal prompt in that folder. A composer project has at a minimum a composer.json file, and a sub-folder labeled vendor . The composer.json file is used to specify what version of which projects your project depends on, the vendor folder is where composer downloads and stores the correct versions of these projects.

To create a composer.json file, one can within the folder of our project type:

 composer init

This starts a wizard that guide you through the process of creating a composer.json file. You don't need to use this wizard, in fact, the file format is relatively simple and a composer.json file can be created/modified in a text editor. Below is a simple example composer.json file suitable for a project that uses Yioop.

 {
     "name": "cpollett/test_composer",
     "description": "used to test how composer works",
     "authors": [
         {
            "name": "Chris Pollett",
             "email": "chris@pollett.org"
         }
     ],
     "require": {
         "seekquarry/yioop": ">=9.0.0"
     }
 }

The name field is the vendor/package you are creating; the description field says what your software does authors says who you are; the repositories field explicitly says where to find the Yioop git repository; and the require field says what versions of which packages your project depends on, in this case, greater than or equal to version 7.0.2 of Yioop. The above composer.json file makes use of the website https://packagist.org/ so that the Yioop repository doesn't explicitly have to be listed. If you don't want to use Packagist, you can add

     "repositories": [
         {
             "type": "vcs",
             "url": "http://seekquarry.com/git/yioop.git"
         }
     ],

to the body of composer.json object above.

Below is a short test file, test.php, that we could create in our project folder. It demonstrates several useful functions in Yioop's library.

<?php
namespace cpollett\test_composer; 
 
use seekquarry\yioop\library as L;
use seekquarry\yioop\library\Library;
use seekquarry\yioop\library\LinearAlgebra as LA;
use seekquarry\yioop\library\FetchUrl;
use seekquarry\yioop\library\PhraseParser;
use seekquarry\yioop\library\CrawlConstants; 
 
require_once "vendor/autoload.php"; 
 
/*
   Since a normal Yioop instance needs a Profile.php file to be generated,
   the following is used to set up Yioop in library mode so you don't need this.
   To enable debugging use Library::init(true);
 */
Library::init(); 
 
// download a collection of web pages and then pretty print
$page_info = FetchUrl::getPages(
    [ [CrawlConstants::URL => "https://www.yahoo.com/"], // could add more urls
    ]
     // we could list more urls to download
);
print_r($page_info); 
 
// stem word or phrases
print_r(PhraseParser::stemTerms("image", 'en-US'));
print_r(PhraseParser::stemTerms("I once knew a jumpy cat", 'en-US'));
print_r(PhraseParser::stemTerms("Allons, enfants de la Patrie,
    Le jour de gloire est arrivé!", 'fr-FR')); 
 
// segment strings into words
print_r(PhraseParser::segmentSegment("从前,在一块遥远的土地上", 'zh-CN')); 
 
//detect language from text
$pinocchio = <<< EOD
Come andò che Maestro Ciliegia, falegname trovò un pezzo di legno che
piangeva e rideva come un bambino. 
 
— C'era una volta.... 
 
— Un re! — diranno subito i miei piccoli lettori. 
 
— No, ragazzi, avete sbagliato. C'era una volta un pezzo di legno. 
 
Non era un legno di lusso, ma un semplice pezzo da catasta, di quelli
che d'inverno si mettono nelle stufe e nei caminetti per accendere il
fuoco e per riscaldare le stanze.
EOD; 
 
$lang = L\guessLocaleFromString($pinocchio);
echo $lang . "\n"; 
 
//make term frequency vector of stemmed terms
$vec = PhraseParser::extractPhrasesAndCount($pinocchio, $lang);
print_r($vec); 
 
// Normalize this vector
$norm = LA::normalize($vec);
print_r($norm);

The namespace line says the namespace of our project, the use lines say what classes and abbreviations for namespaces we will be using. For example, the line

 use seekquarry\yioop\library\PhraseParser

says that if we see the name PhraseParser interpret it as the full namespace and class seekquarry\yioop\library\PhraseParser. The require_once loads the autoloader created by Composer for loading class files of packages it manages. The line:

 Library::init();

is needed to initialize Yioop in Library mode as opposed to being run as a website. The subsequent lines are preceded by a comment sayin what they do. They each illustrate a useful functionality that can be found in Yioop's library.

To get Composer to create the autoload.php file mentioned above as well as install any packages your project depends on, you simply type:

 composer install

Thereafter, if we want to upgrade our project's dependencies to their latest compatible versions we can type:

 composer update

To run the program we just created we can type:

 php test.php

Yioop has a variety of classes which might be useful to your project. The file YIOOP_FOLDER/src/examples/SearchApi.php shows an example of how to use Yioop's search api to do queries on an existing Yioop index. The PhraseParser class also has methods for char-gramming, segmenting, term extraction and counting, etc. The LinearAlgebra class has useful functions for manipulating term vectors. LocaleFunctions has functions useful for handling languages. Yioop's Utility class has functions for various index compression codes. Yioop has classes for text summarization, url manipulation, suffix trees, tries, Bloom filters, compression algorithms, etc, all of which might be useful for project. The online Source Code Documention is a good place to start exploring what classes are available in Yioop.