Difference between revisions of "Main Page"

From GGBN Portal Software
Jump to: navigation, search
m (Adding them in SOLR)
(Adding new search fields)
Line 97: Line 97:
 
         System.gc();
 
         System.gc();
 
     }
 
     }
</code> and add a call to that method in the handleDocuments method:
+
</code> and add a call to that method in the '''handleDocuments''' method:
 
<code>
 
<code>
 
         occurrencesDone.clear();
 
         occurrencesDone.clear();
Line 104: Line 104:
 
</code>
 
</code>
 
*existing documents can not be updated in SOLR in our case (only if everything was "stored"). So first, delete the old data from the core. Depending on how many records there are, you might have to delete the data for a specific repository only solrServer:solrPort/solr/coreName_SEARCH_OR_PREVIEW_OR_DETAILS/update?stream.body=<delete><query>YOUR_LIMITATION_FIELD:*</query></delete>&commit=true This "limitation_field" must be an indexed value in the core. If you don't have a lot of data, you can put <query>*:*</query> as the generation of the new SOLR index will be fast enough (5-10min per Core for 180.000 records).
 
*existing documents can not be updated in SOLR in our case (only if everything was "stored"). So first, delete the old data from the core. Depending on how many records there are, you might have to delete the data for a specific repository only solrServer:solrPort/solr/coreName_SEARCH_OR_PREVIEW_OR_DETAILS/update?stream.body=<delete><query>YOUR_LIMITATION_FIELD:*</query></delete>&commit=true This "limitation_field" must be an indexed value in the core. If you don't have a lot of data, you can put <query>*:*</query> as the generation of the new SOLR index will be fast enough (5-10min per Core for 180.000 records).
 +
*per default, it will index this new field for each datasource. If it's still missing at the end for some datasources, or if you only want to do it for specific datasources, edit the '''main''' method <code>for (int i=0;i<166;i++)</code>
  
 
=Debug=
 
=Debug=
 
If something does not work as expected, have a look in the catalina.out file (generated by Tomcat) on the machine where SOLR is running. You will see the last queries sent to SOLR at the end of the file.
 
If something does not work as expected, have a look in the catalina.out file (generated by Tomcat) on the machine where SOLR is running. You will see the last queries sent to SOLR at the end of the file.
 
The YII file runtime/log/app.log might also help, and is located in the yii folder on the machine running the portal (ie. advanced/runtime....).
 
The YII file runtime/log/app.log might also help, and is located in the yii folder on the machine running the portal (ie. advanced/runtime....).

Revision as of 13:44, 18 January 2016

Welcome to the Wiki of the GGBN Portal Software

Further information about the software will follow soon.


Presentation

TBD

Workflow

TBD

Installation

You will need

  • Apache (tested with Apache/2.4.7 (Ubuntu), Apache/2.2.22 (Debian), Apache/2.4.10 (Win64)) with libapache2-mod-php5
  • PHP5 (tested with 5.5.9, 5.5.30 and 5.5.15). You also might need to install php5-xsl, php5-gd and php5-mysql.
  • a SOLR instance (tested with Apache SOLR 4.9.0). Contact us for getting the appropriate SOLR configuration files.
  • a MySQL database (tested with MySQL 5.5.46). It will contain data harvested thanks to B-HIT

Yii (version 2) will be fully contained in the subversion code of the portal software.

Copy a working version from the svn

Create a destination folder, for the rest of this documentation we will choose "my_portal" in your web-folder (ie. /var/www/, or *WINPATH*).
Get http://ww2.biocase.org/svn/dnabank/Dnabank_Portal/ggbn_portal/trunk/ into "my_portal".

Configuration

TBD yii, Apache, rights, SOLR (SOLR config files also & Java)
First, let say we want to access the portal under the URL http://localhost/ggbn_portal.

  • Check that the Apache User has writing/editing rights in the whole "my_portal" folder.
  • Edit the file "my_portal/frontend/config/main.php".
    • Configure the portal URL: $baseUrl = str_replace ( '/SUBDIR_AFTER_YOUR_WEB_FOLDER/my_portal/frontend/web', '/ggbn_portal', (new Request ())->getBaseUrl () );
    • Configure the MySQL database credentials
                                               'db' => [
                                               'class' => '\yii\db\Connection',
                                               'dsn' => 'mysql:host=IP;dbname=DBNAME',
                                               'username' => 'USER',
                                               'password' => 'PASS',
                                               'charset' => 'utf8' 
                               ],
  • Edit the file "my_portal/frontend/config/params.php".
    • e-mails
      • Configure the admin e-mail 'adminEmail' => 'ADMIN@DOMAIN',. The admin will get error/bug messages.
      • Configure the support e-mail(s) 'supportEmail' => ['MAIL1','MAIL2'..],.
      • Configure the feedback e-mail(s) 'feedbackMail' => ['MAIL1','MAIL2'..],. They will get the messages sent with the feedback function.
      • Configure the no-reply-mail (who is supposed to sent e-mail(s) to the users) 'noreplyMail'=>'MAIL'
    • Configure the location of the DarwinCore Archives (harvested with B-HIT) <code>'dwcArchivesPath' => 'PATH_TO_THE_PARENT_FOLDER' ,. Check the access rights! (read is enough)
    • Configure the location(root) of your SOLR instance 'solrhost'=> 'SOLR_HOST' and the SOLR core names

                                               $searchname="SEARCH_CORE";
                                               $previewname="PREVIEW_CORE";
                                               $detailsname="DETAILS_CORE";

    • Set the temporary folder 'tmpFolder'=>'TEMP_FOLDER'. Check the access rights! (write is needed)
    • Configure the options
      • 'useLogin'=>true_OR_false: is the login enabled or disabled
      • 'noBack'=>true_OR_false: a special backbone is used
      • 'shopping'=>true_OR_false: the shopping system is enabled or disabled
      • 'useLogin'=>true_OR_false: is the login enabled or disabled
      • 'newsService'=>true_OR_false: uses the GGBN news services - if it has to be used with a different wiki news, edit the SiteController.py
      • 'displayMore'=>true_OR_false: displays the categories with internal links on the details page
      • 'displayExternals'=>true_OR_false: displays the categories with external links (GBIF, BOLD, NCBI...) on the details page
      • 'ABCDdownloable'=>true_OR_false: ABCD files are downloadable or not
      • 'useAnnosys'=>true_OR_false: link to annosys (annotation system) on the details page
      • 'annosysAP'=>'https://annosys.bgbm.fu-berlin.de/AnnoSys/AnnoSys?providerURL=': root URL for Annosys
      • 'siteName'=>'my_portal',: the portal name (used in the URL)
      • 'siteTitle'=>'GGBN Portal',: the default site name/title


Personalisation

If you want to change the icon (favicon), it is located in frontend/web.
If you want to change the slider images on the home page, they are located in frontend/web/images/slider. Then you will have to modify the view in frontend/view/site/index.php to load the pictures you chose.
If you want to change the menu icons, they are located in frontend/web/images/icons.

The CSS and JS files are also in the frontend/web folder.

Adding new search fields

Adding them in SOLR

  • edit the schema.xml file in SOLR and add the new fields you want to have. If you want to search for them, edit the file for the search core *_search/conf/schema.xml. If wou want to display them in the preview, edit the file for the preview core *_preview/conf/schema.xml. If you need it for the details page, edit the file for the details core *_details/conf/schema.xml. Generally: if you want to search for it (search core), the field has to be indexed, but not mandatory stored. If you want to display it (preview/details), it has to be stored but not indexed.
  • restart Tomcat
  • edit the Java file corresponding to the core that need to be updated- for example, if you want to index or store the recordbasis, add the following method

private void addRecordbasis(int datasourceid) throws ClassNotFoundException, SQLException {
       System.out.println("RUN RECORDBASIS ");
       if(tripleidstoreids.size()>0){
           String queryTID =
                   "SELECT distinct fk_tripleidstoreid as tripleidstoreid,  "
                           + "IFNULL(recordbasis,'N/A') as recordbasis "
                           + " FROM recordbasis "
                           + " JOIN occurrence ON recordbasisid = fk_recordbasisid "
                           + " WHERE fk_tripleidstoreid in ("+StringUtils.join(tripleidstoreids,", ")+") "
                           + " AND fk_datasourceid ="+datasourceid
                           + " ORDER BY fk_tripleidstoreid ";
           update( queryTID,tripleidstoreids.size());
       }
       System.gc();
   }

and add a call to that method in the handleDocuments method:

       occurrencesDone.clear();
       multivalueForbidden=false;
       addRecordbasis(datasourceid );

  • existing documents can not be updated in SOLR in our case (only if everything was "stored"). So first, delete the old data from the core. Depending on how many records there are, you might have to delete the data for a specific repository only solrServer:solrPort/solr/coreName_SEARCH_OR_PREVIEW_OR_DETAILS/update?stream.body=<delete><query>YOUR_LIMITATION_FIELD:*</query></delete>&commit=true This "limitation_field" must be an indexed value in the core. If you don't have a lot of data, you can put <query>*:*</query> as the generation of the new SOLR index will be fast enough (5-10min per Core for 180.000 records).
  • per default, it will index this new field for each datasource. If it's still missing at the end for some datasources, or if you only want to do it for specific datasources, edit the main method for (int i=0;i<166;i++)

Debug

If something does not work as expected, have a look in the catalina.out file (generated by Tomcat) on the machine where SOLR is running. You will see the last queries sent to SOLR at the end of the file. The YII file runtime/log/app.log might also help, and is located in the yii folder on the machine running the portal (ie. advanced/runtime....).