GGBN Portal Software - User contributions [en]

Main Page

2016-09-22T12:53:44Z

GabiDroege: /* Configuration */

'''Welcome to the Wiki of the GGBN Portal Software'''

=Introduction=
The GGBN Portal Software (GPS) has been developed by the Botanic Garden and Botanical Museum Berlin-Dahlem for the Global Genome Biodiversity Network ([http://www.ggbn.org GGBN]). The project was funded by the German Research Foundation (DFG (01/2014 - 03/2016)). The software is written in php using the [http://www.yiiframework.com/ Yii framework] and fully open source. The GPS allows costumization and therefore can be used for other special interest networks too. It is optimized for an underlying SOLR instance, data are harvested with [http://wiki.bgbm.org/bhit/ B-HIT].

=Workflow=
[[File:Workflow GPS.jpg|500px]]

Data are provided through BioCASe and IPT. The harvester [http://wiki.bgbm.org/bhit B-HIT] stores all original data in a MySQL database. To find out more about B-HIT please check out the [http://wiki.bgbm.org/bhit B-HIT Wiki]. After harvesting data are cleaned and can be matched against different taxonomic backbones. A SOLR instance is created on top of the MySQL database. The GPS itself requires access to the SOLR index as well as the MySQL index.

=Installation=

==You will need==
* Apache (tested with Apache/2.4.7 (Ubuntu), Apache/2.2.22 (Debian), Apache/2.4.10 (Win64)) with libapache2-mod-php5
* PHP5 (from 5.5! tested with 5.5.9, 5.5.30 and 5.5.15). You also might need to install php5-xsl, php5-gd and php5-mysql.
* a SOLR instance (tested with Apache SOLR 4.9.0). Contact us for getting the appropriate SOLR configuration files.
* a MySQL database (tested with MySQL 5.5.46). It will contain data harvested with [http://wiki.bgbm.org/bhit/index.php/Main_Page B-HIT]

Yii (version 2) will be fully contained in the subversion code of the portal software.

==Copy a working version from the svn ==
Create a destination folder, for the rest of this documentation we will choose "my_portal" in your web-folder (ie. /var/www/, or *WINPATH*). 
Get http://ww2.biocase.org/svn/dnabank/Dnabank_Portal/ggbn_portal/trunk/ into "my_portal".

=Configuration=
''TBD''
''yii, Apache, rights, SOLR (SOLR config files also & Java)'' 
First, let say we want to access the portal under the URL http://localhost/ggbn_portal.

*Check that the Apache User has writing/editing rights in the whole "my_portal" folder.
*Edit the file "my_portal/frontend/config/main.php".
**Configure the portal URL: <code>$baseUrl = str_replace ( '/SUBDIR_AFTER_YOUR_WEB_FOLDER/my_portal/frontend/web', '/ggbn_portal', (new Request ())->getBaseUrl () );</code>
**Configure the MySQL database credentials <code>
'db' => [
'class' => '\yii\db\Connection',
'dsn' => 'mysql:host=IP;dbname=DBNAME',
'username' => 'USER',
'password' => 'PASS',
'charset' => 'utf8'
],</code>
*Edit the file "my_portal/frontend/config/params.php".
**e-mails
***Configure the admin e-mail <code>'adminEmail' => 'ADMIN@DOMAIN',</code>. The admin will get error/bug messages.
***Configure the support e-mail(s) <code>'supportEmail' => ['MAIL1','MAIL2'..],</code>.
***Configure the feedback e-mail(s) <code>'feedbackMail' => ['MAIL1','MAIL2'..],</code>. They will get the messages sent with the feedback function.
***Configure the no-reply-mail (who is supposed to sent e-mail(s) to the users) <code>'noreplyMail'=>'MAIL'
**Configure the location of the DarwinCore Archives (harvested with B-HIT) <code>'dwcArchivesPath' => 'PATH_TO_THE_PARENT_FOLDER' ,</code>. Check the access rights! (read is enough), should be an URL if you want to use Annosys
**Configure the location(root) of your SOLR instance <code>'solrhost'=> 'SOLR_HOST'</code> and the SOLR core names
<code>
$searchname="SEARCH_CORE";
$previewname="PREVIEW_CORE";
$detailsname="DETAILS_CORE";
</code>
There is also a core for the statistics pages (<code>ggbn_stats</code>).
**Set the temporary folder <code>'tmpFolder'=>'TEMP_FOLDER'</code>. Check the access rights! (write is needed)
**Configure the options
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'noBack'=>true_OR_false</code>: a special backbone is used
***<code>'shopping'=>true_OR_false</code>: the shopping system is enabled or disabled
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'newsService'=>true_OR_false</code>: uses the GGBN news services - if it has to be used with a different wiki news, edit the SiteController.py
***<code>'viewsCountsService'=>true_OR_false</code>: uses the GGBN specific views containing some counts (number of species, dna, ...). Look in the '''GGBN specific configuration''' for the SQL statements.
***<code>'displayMore'=>true_OR_false</code>: displays the categories with internal links on the details page
***<code>'displayExternals'=>true_OR_false</code>: displays the categories with external links (GBIF, BOLD, NCBI...) on the details page
***<code>'ABCDdownloable'=>true_OR_false</code>: ABCD files are downloadable or not
***<code>'useAnnosys'=>true_OR_false</code>: link to annosys (annotation system) on the details page
***<code>'annosysAP'=>'https://annosys.bgbm.fu-berlin.de/AnnoSys/AnnoSys?providerURL='</code>: root URL for Annosys
***<code>'siteName'=>'my_portal',</code>: the portal name (used in the URL)
***<code>'siteTitle'=>'GGBN Portal',</code>: the default site name/title
***<code>'catalogNumberClickable'=>true,</code>: if the catalogNumber/UnitID can be clickable on the details page

*Edit the Apache configuration file, for example:
<pre>
Alias /ggbn_portal "/var/www/my_portal/frontend/web"
<Directory "/var/www/my_portal/frontend/web">
Options All
AllowOverride All
</Directory>
</pre>

==Production/Debug Mode==
Switching Production/Debug Mode can be done in my_portal/web/index.php

=Personalisation=
If you want to change the icon (favicon), it is located in frontend/web. 
If you want to change the slider images on the home page, they are located in frontend/web/images/slider. Then you will have to modify the view in frontend/view/site/index.php to load the pictures you chose. 
If you want to change the menu icons, they are located in frontend/web/images/icons. 
Navigation menu is located in frontend/views/layouts/main.php 
Start page, news section (wiki api) is located in frontend/controllers/SiteController.php 
 
The CSS and JS files are also in the frontend/web folder.

=Adding new search fields or fields to display=
==Adding them in SOLR==
*edit the schema.xml file in SOLR and add the new fields you want to have. If you want to search for them, edit the file for the search core *_search/conf/schema.xml. If wou want to display them in the preview, edit the file for the preview core *_preview/conf/schema.xml. If you need it for the details page, edit the file for the details core *_details/conf/schema.xml. Generally: if you want to search for it (search core), the field has to be indexed, but not mandatory stored. If you want to display it (preview/details), it has to be stored but not indexed. The tripleidstoreid or the occurrenceid *always* has to be indexed.
*restart Tomcat
'''Important ''': always restart Tomcat (service tomcat8 restart) when you modify the schema.xml file.
*edit the Java file corresponding to the core that need to be updated- for example, if you want to index or store the recordbasis, add the following method
<code>
private void addRecordbasis(int datasourceid) throws ClassNotFoundException, SQLException {
System.out.println("RUN RECORDBASIS ");
if(tripleidstoreids.size()>0){
String queryTID =
"SELECT distinct fk_tripleidstoreid as tripleidstoreid, "
+ "IFNULL(recordbasis,'N/A') as recordbasis "
+ " FROM recordbasis "
+ " JOIN occurrence ON recordbasisid = fk_recordbasisid "
+ " WHERE fk_tripleidstoreid in ("+StringUtils.join(tripleidstoreids,", ")+") "
+ " AND fk_datasourceid ="+datasourceid
+ " ORDER BY fk_tripleidstoreid ";
update( queryTID,tripleidstoreids.size());
}
System.gc();
}
</code> and add a call to that method in the '''handleDocuments''' method:
<code>
occurrencesDone.clear();
multivalueForbidden=false;
addRecordbasis(datasourceid );
</code>
*existing documents can not be updated in SOLR in our case (only if everything was "stored"). So first, delete the old data from the core. Depending on how many records there are, you might have to delete the data for a specific repository only solrServer:solrPort/solr/coreName_SEARCH_OR_PREVIEW_OR_DETAILS/update?stream.body=<delete><query>YOUR_LIMITATION_FIELD:*</query></delete>&commit=true This "limitation_field" must be an indexed value in the core. If you don't have a lot of data, you can put <query>*:*</query> as the generation of the new SOLR index will be fast enough (5-10min per Core for 180.000 records).
*per default, it will index this new field for each datasource. If it's still missing at the end for some datasources, or if you only want to do it for specific datasources, edit the '''main''' method <code>for (int i=0;i<166;i++)</code>

==Adding them in the portal==
*Edit the SearchController file (controllers)(additions can be made in the CommonSearchController, deletion only in the specific SearchController if you have several portals running in parallel with the same installation)
**if it's a dropdownlist, add the name of the field in the variable <code>$knownLParams</code>.
**if it's a simple term or a suggestion list, add the name of the field in the variable <code>$knownParams</code>.
*Edit the SearchForm file (models)(same rule as above), edit the variable <code>$_parameters</code> - it can be a text, a select(dropdownlist), a suggest, a radio or a range. Also add the public variable with the same name (ie. <code>public $sampleavailability;</code> and
<code>
'sampleavailability' => [
'type' => 'radio',
'name' => 'Sample availability (loan)',
'config' => [
'size' => 285,
'maxlength' => 150,
'name' => 'sampleavailability'
],
'hideable' => 'true'
]
</code>. It will be automatically displayed in the search form. The '''hideable''' parameter specifies if it is displayed per default or if it is only in the accordion.
Eventually, edit the '''attribteLabels''' function such as <code>'sampleavailability' => 'Sample availability (loan)'</code>
*Edit the listView.php (views) file if it has to be displayed here, or the recordViewTabs.php if it has to be displayed on the details page. Here again, add it to the $suggestFields or $defaultFields or $selectFields or $buttonFields, depending of its type.
*Edit the file CreateFormManager(vendor) only if it's a radio button (look how it's done for cites and sampleavailability).
*Edit the file SOLRQueryManager(vendor) - for example, if it has to look in the SOLR case-unsensitive value, do as for
<code>
case 'fullScientificName' :
return $field . '_nc:' . $value . '';
break;
</code> If it's case sensitive,
<code>
case 'collectioncode' :
return $field . ':"' . $value . '"';
break;
</code>

=Running multiple GPS portals in parallel=
[[File:gps_folders.jpg|thumb|left|300px|Example of two portals running in parallel, specific files are located in "ggbnportal" and "worldfloraonline". All other folders are used by both portal instances]]

*in COMMON_PORTALFILES/common/config/bootstrap.php: add the new portal folder here; required for Yii import routines
*in COMMON_PORTALFILES/environnments/index.php: add the new portal folder here; the name of the folders that will be exposed and therefore where Yii should be authorized to write logs and handle the CSS files
*COMMON_PORTALFILES/NEW_Portal/config/params.php and main.php: change database, SOLR and all other parameters

=Debug=
If something does not work as expected, have a look in the catalina.out file (generated by Tomcat) on the machine where SOLR is running. If Tomcat is running through XAMPP, you might have a part of the logs in the catalina.bat window. You will see the last queries sent to SOLR at the end of the file.
The YII file runtime/log/app.log might also help, and is located in the yii folder on the machine running the portal (ie. advanced/runtime....).

Main Page

2016-09-22T12:50:39Z

GabiDroege: /* Running multiple GPS portals in parallel */

'''Welcome to the Wiki of the GGBN Portal Software'''

=Introduction=
The GGBN Portal Software (GPS) has been developed by the Botanic Garden and Botanical Museum Berlin-Dahlem for the Global Genome Biodiversity Network ([http://www.ggbn.org GGBN]). The project was funded by the German Research Foundation (DFG (01/2014 - 03/2016)). The software is written in php using the [http://www.yiiframework.com/ Yii framework] and fully open source. The GPS allows costumization and therefore can be used for other special interest networks too. It is optimized for an underlying SOLR instance, data are harvested with [http://wiki.bgbm.org/bhit/ B-HIT].

=Workflow=
[[File:Workflow GPS.jpg|500px]]

Data are provided through BioCASe and IPT. The harvester [http://wiki.bgbm.org/bhit B-HIT] stores all original data in a MySQL database. To find out more about B-HIT please check out the [http://wiki.bgbm.org/bhit B-HIT Wiki]. After harvesting data are cleaned and can be matched against different taxonomic backbones. A SOLR instance is created on top of the MySQL database. The GPS itself requires access to the SOLR index as well as the MySQL index.

=Installation=

==You will need==
* Apache (tested with Apache/2.4.7 (Ubuntu), Apache/2.2.22 (Debian), Apache/2.4.10 (Win64)) with libapache2-mod-php5
* PHP5 (from 5.5! tested with 5.5.9, 5.5.30 and 5.5.15). You also might need to install php5-xsl, php5-gd and php5-mysql.
* a SOLR instance (tested with Apache SOLR 4.9.0). Contact us for getting the appropriate SOLR configuration files.
* a MySQL database (tested with MySQL 5.5.46). It will contain data harvested with [http://wiki.bgbm.org/bhit/index.php/Main_Page B-HIT]

Yii (version 2) will be fully contained in the subversion code of the portal software.

==Copy a working version from the svn ==
Create a destination folder, for the rest of this documentation we will choose "my_portal" in your web-folder (ie. /var/www/, or *WINPATH*). 
Get http://ww2.biocase.org/svn/dnabank/Dnabank_Portal/ggbn_portal/trunk/ into "my_portal".

=Configuration=
''TBD''
''yii, Apache, rights, SOLR (SOLR config files also & Java)'' 
First, let say we want to access the portal under the URL http://localhost/ggbn_portal.

*Check that the Apache User has writing/editing rights in the whole "my_portal" folder.
*Edit the file "my_portal/frontend/config/main.php".
**Configure the portal URL: <code>$baseUrl = str_replace ( '/SUBDIR_AFTER_YOUR_WEB_FOLDER/my_portal/frontend/web', '/ggbn_portal', (new Request ())->getBaseUrl () );</code>
**Configure the MySQL database credentials <code>
'db' => [
'class' => '\yii\db\Connection',
'dsn' => 'mysql:host=IP;dbname=DBNAME',
'username' => 'USER',
'password' => 'PASS',
'charset' => 'utf8'
],</code>
*Edit the file "my_portal/frontend/config/params.php".
**e-mails
***Configure the admin e-mail <code>'adminEmail' => 'ADMIN@DOMAIN',</code>. The admin will get error/bug messages.
***Configure the support e-mail(s) <code>'supportEmail' => ['MAIL1','MAIL2'..],</code>.
***Configure the feedback e-mail(s) <code>'feedbackMail' => ['MAIL1','MAIL2'..],</code>. They will get the messages sent with the feedback function.
***Configure the no-reply-mail (who is supposed to sent e-mail(s) to the users) <code>'noreplyMail'=>'MAIL'
**Configure the location of the DarwinCore Archives (harvested with B-HIT) <code>'dwcArchivesPath' => 'PATH_TO_THE_PARENT_FOLDER' ,</code>. Check the access rights! (read is enough), should be an URL if you want to use Annosys
**Configure the location(root) of your SOLR instance <code>'solrhost'=> 'SOLR_HOST'</code> and the SOLR core names
<code>
$searchname="SEARCH_CORE";
$previewname="PREVIEW_CORE";
$detailsname="DETAILS_CORE";
</code>
There is also a core for the statistics pages (<code>ggbn_stats</code>).
**Set the temporary folder <code>'tmpFolder'=>'TEMP_FOLDER'</code>. Check the access rights! (write is needed)
**Configure the options
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'noBack'=>true_OR_false</code>: a special backbone is used
***<code>'shopping'=>true_OR_false</code>: the shopping system is enabled or disabled
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'newsService'=>true_OR_false</code>: uses the GGBN news services - if it has to be used with a different wiki news, edit the SiteController.py
***<code>'viewsCountsService'=>true_OR_false</code>: uses the GGBN specific views containing some counts (number of species, dna, ...). Look in the '''GGBN specific configuration''' for the SQL statements.
***<code>'displayMore'=>true_OR_false</code>: displays the categories with internal links on the details page
***<code>'displayExternals'=>true_OR_false</code>: displays the categories with external links (GBIF, BOLD, NCBI...) on the details page
***<code>'ABCDdownloable'=>true_OR_false</code>: ABCD files are downloadable or not
***<code>'useAnnosys'=>true_OR_false</code>: link to annosys (annotation system) on the details page
***<code>'annosysAP'=>'https://annosys.bgbm.fu-berlin.de/AnnoSys/AnnoSys?providerURL='</code>: root URL for Annosys
***<code>'siteName'=>'my_portal',</code>: the portal name (used in the URL)
***<code>'siteTitle'=>'GGBN Portal',</code>: the default site name/title
***<code>'catalogNumberClickable'=>true,</code>: if the catalogNumber/UnitID can be clickable on the details page

*Edit the Apache configuration file, for example:
<pre>
Alias /ggbn_portal "/var/www/my_portal/frontend/web"
<Directory "/var/www/my_portal/frontend/web">
Options All
AllowOverride All
</Directory>
</pre>

=Personalisation=
If you want to change the icon (favicon), it is located in frontend/web. 
If you want to change the slider images on the home page, they are located in frontend/web/images/slider. Then you will have to modify the view in frontend/view/site/index.php to load the pictures you chose. 
If you want to change the menu icons, they are located in frontend/web/images/icons. 
Navigation menu is located in frontend/views/layouts/main.php 
Start page, news section (wiki api) is located in frontend/controllers/SiteController.php 
 
The CSS and JS files are also in the frontend/web folder.

=Adding new search fields or fields to display=
==Adding them in SOLR==
*edit the schema.xml file in SOLR and add the new fields you want to have. If you want to search for them, edit the file for the search core *_search/conf/schema.xml. If wou want to display them in the preview, edit the file for the preview core *_preview/conf/schema.xml. If you need it for the details page, edit the file for the details core *_details/conf/schema.xml. Generally: if you want to search for it (search core), the field has to be indexed, but not mandatory stored. If you want to display it (preview/details), it has to be stored but not indexed. The tripleidstoreid or the occurrenceid *always* has to be indexed.
*restart Tomcat
'''Important ''': always restart Tomcat (service tomcat8 restart) when you modify the schema.xml file.
*edit the Java file corresponding to the core that need to be updated- for example, if you want to index or store the recordbasis, add the following method
<code>
private void addRecordbasis(int datasourceid) throws ClassNotFoundException, SQLException {
System.out.println("RUN RECORDBASIS ");
if(tripleidstoreids.size()>0){
String queryTID =
"SELECT distinct fk_tripleidstoreid as tripleidstoreid, "
+ "IFNULL(recordbasis,'N/A') as recordbasis "
+ " FROM recordbasis "
+ " JOIN occurrence ON recordbasisid = fk_recordbasisid "
+ " WHERE fk_tripleidstoreid in ("+StringUtils.join(tripleidstoreids,", ")+") "
+ " AND fk_datasourceid ="+datasourceid
+ " ORDER BY fk_tripleidstoreid ";
update( queryTID,tripleidstoreids.size());
}
System.gc();
}
</code> and add a call to that method in the '''handleDocuments''' method:
<code>
occurrencesDone.clear();
multivalueForbidden=false;
addRecordbasis(datasourceid );
</code>
*existing documents can not be updated in SOLR in our case (only if everything was "stored"). So first, delete the old data from the core. Depending on how many records there are, you might have to delete the data for a specific repository only solrServer:solrPort/solr/coreName_SEARCH_OR_PREVIEW_OR_DETAILS/update?stream.body=<delete><query>YOUR_LIMITATION_FIELD:*</query></delete>&commit=true This "limitation_field" must be an indexed value in the core. If you don't have a lot of data, you can put <query>*:*</query> as the generation of the new SOLR index will be fast enough (5-10min per Core for 180.000 records).
*per default, it will index this new field for each datasource. If it's still missing at the end for some datasources, or if you only want to do it for specific datasources, edit the '''main''' method <code>for (int i=0;i<166;i++)</code>

==Adding them in the portal==
*Edit the SearchController file (controllers)(additions can be made in the CommonSearchController, deletion only in the specific SearchController if you have several portals running in parallel with the same installation)
**if it's a dropdownlist, add the name of the field in the variable <code>$knownLParams</code>.
**if it's a simple term or a suggestion list, add the name of the field in the variable <code>$knownParams</code>.
*Edit the SearchForm file (models)(same rule as above), edit the variable <code>$_parameters</code> - it can be a text, a select(dropdownlist), a suggest, a radio or a range. Also add the public variable with the same name (ie. <code>public $sampleavailability;</code> and
<code>
'sampleavailability' => [
'type' => 'radio',
'name' => 'Sample availability (loan)',
'config' => [
'size' => 285,
'maxlength' => 150,
'name' => 'sampleavailability'
],
'hideable' => 'true'
]
</code>. It will be automatically displayed in the search form. The '''hideable''' parameter specifies if it is displayed per default or if it is only in the accordion.
Eventually, edit the '''attribteLabels''' function such as <code>'sampleavailability' => 'Sample availability (loan)'</code>
*Edit the listView.php (views) file if it has to be displayed here, or the recordViewTabs.php if it has to be displayed on the details page. Here again, add it to the $suggestFields or $defaultFields or $selectFields or $buttonFields, depending of its type.
*Edit the file CreateFormManager(vendor) only if it's a radio button (look how it's done for cites and sampleavailability).
*Edit the file SOLRQueryManager(vendor) - for example, if it has to look in the SOLR case-unsensitive value, do as for
<code>
case 'fullScientificName' :
return $field . '_nc:' . $value . '';
break;
</code> If it's case sensitive,
<code>
case 'collectioncode' :
return $field . ':"' . $value . '"';
break;
</code>

=Running multiple GPS portals in parallel=
[[File:gps_folders.jpg|thumb|left|300px|Example of two portals running in parallel, specific files are located in "ggbnportal" and "worldfloraonline". All other folders are used by both portal instances]]

*in COMMON_PORTALFILES/common/config/bootstrap.php: add the new portal folder here; required for Yii import routines
*in COMMON_PORTALFILES/environnments/index.php: add the new portal folder here; the name of the folders that will be exposed and therefore where Yii should be authorized to write logs and handle the CSS files
*COMMON_PORTALFILES/NEW_Portal/config/params.php and main.php: change database, SOLR and all other parameters

=Debug=
If something does not work as expected, have a look in the catalina.out file (generated by Tomcat) on the machine where SOLR is running. If Tomcat is running through XAMPP, you might have a part of the logs in the catalina.bat window. You will see the last queries sent to SOLR at the end of the file.
The YII file runtime/log/app.log might also help, and is located in the yii folder on the machine running the portal (ie. advanced/runtime....).

Search Feature

2016-08-30T07:46:47Z

GabiDroege: /* Retrieved from Common folder */

=Search Form=
* 2 main controllers: Search & WebsiteSearch
** Local WebsitesearchController extends CommonWebsitesearchController
** Local SearchController

==views/search/search.php==
Contains some of the display specifications
* website search tabs (e.g. GGBN) can be switched on/off in config/params.php
* full code identical for WFO and GGBN; can be customized, but for updates this script should be kept generic

==models/SearchForm.php==
* extends CommonSearchForm, but contains a lot of customized styling (e.g. labels of all form fields)

==Retrieved from Common folder==
* views/search/mapSearch.php
* views/search/searchFieldSelection.php

Search Feature

2016-08-30T07:30:12Z

GabiDroege: Created page with "=Search Form= * 2 main controllers: Search & WebsiteSearch ** Local WebsitesearchController extends CommonWebsitesearchController ** Local SearchController ==views/search/se..."

Main Page

2016-03-24T09:00:11Z

GabiDroege: /* Running multiple GPS portals in parallel */

'''Welcome to the Wiki of the GGBN Portal Software'''

=Introduction=
The GGBN Portal Software (GPS) has been developed by the Botanic Garden and Botanical Museum Berlin-Dahlem for the Global Genome Biodiversity Network ([http://www.ggbn.org GGBN]). The project was funded by the German Research Foundation (DFG (01/2014 - 03/2016)). The software is written in php using the [http://www.yiiframework.com/ Yii framework] and fully open source. The GPS allows costumization and therefore can be used for other special interest networks too. It is optimized for an underlying SOLR instance, data are harvested with [http://wiki.bgbm.org/bhit/ B-HIT].

=Workflow=
[[File:Workflow GPS.jpg|500px]]

Data are provided through BioCASe and IPT. The harvester [http://wiki.bgbm.org/bhit B-HIT] stores all original data in a MySQL database. To find out more about B-HIT please check out the [http://wiki.bgbm.org/bhit B-HIT Wiki]. After harvesting data are cleaned and can be matched against different taxonomic backbones. A SOLR instance is created on top of the MySQL database. The GPS itself requires access to the SOLR index as well as the MySQL index.

=Installation=

==You will need==
* Apache (tested with Apache/2.4.7 (Ubuntu), Apache/2.2.22 (Debian), Apache/2.4.10 (Win64)) with libapache2-mod-php5
* PHP5 (from 5.5! tested with 5.5.9, 5.5.30 and 5.5.15). You also might need to install php5-xsl, php5-gd and php5-mysql.
* a SOLR instance (tested with Apache SOLR 4.9.0). Contact us for getting the appropriate SOLR configuration files.
* a MySQL database (tested with MySQL 5.5.46). It will contain data harvested with [http://wiki.bgbm.org/bhit/index.php/Main_Page B-HIT]

Yii (version 2) will be fully contained in the subversion code of the portal software.

==Copy a working version from the svn ==
Create a destination folder, for the rest of this documentation we will choose "my_portal" in your web-folder (ie. /var/www/, or *WINPATH*). 
Get http://ww2.biocase.org/svn/dnabank/Dnabank_Portal/ggbn_portal/trunk/ into "my_portal".

=Configuration=
''TBD''
''yii, Apache, rights, SOLR (SOLR config files also & Java)'' 
First, let say we want to access the portal under the URL http://localhost/ggbn_portal.

*Check that the Apache User has writing/editing rights in the whole "my_portal" folder.
*Edit the file "my_portal/frontend/config/main.php".
**Configure the portal URL: <code>$baseUrl = str_replace ( '/SUBDIR_AFTER_YOUR_WEB_FOLDER/my_portal/frontend/web', '/ggbn_portal', (new Request ())->getBaseUrl () );</code>
**Configure the MySQL database credentials <code>
'db' => [
'class' => '\yii\db\Connection',
'dsn' => 'mysql:host=IP;dbname=DBNAME',
'username' => 'USER',
'password' => 'PASS',
'charset' => 'utf8'
],</code>
*Edit the file "my_portal/frontend/config/params.php".
**e-mails
***Configure the admin e-mail <code>'adminEmail' => 'ADMIN@DOMAIN',</code>. The admin will get error/bug messages.
***Configure the support e-mail(s) <code>'supportEmail' => ['MAIL1','MAIL2'..],</code>.
***Configure the feedback e-mail(s) <code>'feedbackMail' => ['MAIL1','MAIL2'..],</code>. They will get the messages sent with the feedback function.
***Configure the no-reply-mail (who is supposed to sent e-mail(s) to the users) <code>'noreplyMail'=>'MAIL'
**Configure the location of the DarwinCore Archives (harvested with B-HIT) <code>'dwcArchivesPath' => 'PATH_TO_THE_PARENT_FOLDER' ,</code>. Check the access rights! (read is enough), should be an URL if you want to use Annosys
**Configure the location(root) of your SOLR instance <code>'solrhost'=> 'SOLR_HOST'</code> and the SOLR core names
<code>
$searchname="SEARCH_CORE";
$previewname="PREVIEW_CORE";
$detailsname="DETAILS_CORE";
</code>
There is also a core for the statistics pages (<code>ggbn_stats</code>).
**Set the temporary folder <code>'tmpFolder'=>'TEMP_FOLDER'</code>. Check the access rights! (write is needed)
**Configure the options
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'noBack'=>true_OR_false</code>: a special backbone is used
***<code>'shopping'=>true_OR_false</code>: the shopping system is enabled or disabled
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'newsService'=>true_OR_false</code>: uses the GGBN news services - if it has to be used with a different wiki news, edit the SiteController.py
***<code>'viewsCountsService'=>true_OR_false</code>: uses the GGBN specific views containing some counts (number of species, dna, ...). Look in the '''GGBN specific configuration''' for the SQL statements.
***<code>'displayMore'=>true_OR_false</code>: displays the categories with internal links on the details page
***<code>'displayExternals'=>true_OR_false</code>: displays the categories with external links (GBIF, BOLD, NCBI...) on the details page
***<code>'ABCDdownloable'=>true_OR_false</code>: ABCD files are downloadable or not
***<code>'useAnnosys'=>true_OR_false</code>: link to annosys (annotation system) on the details page
***<code>'annosysAP'=>'https://annosys.bgbm.fu-berlin.de/AnnoSys/AnnoSys?providerURL='</code>: root URL for Annosys
***<code>'siteName'=>'my_portal',</code>: the portal name (used in the URL)
***<code>'siteTitle'=>'GGBN Portal',</code>: the default site name/title
***<code>'catalogNumberClickable'=>true,</code>: if the catalogNumber/UnitID can be clickable on the details page

*Edit the Apache configuration file, for example:
<pre>
Alias /ggbn_portal "/var/www/my_portal/frontend/web"
<Directory "/var/www/my_portal/frontend/web">
Options All
AllowOverride All
</Directory>
</pre>

=Personalisation=
If you want to change the icon (favicon), it is located in frontend/web. 
If you want to change the slider images on the home page, they are located in frontend/web/images/slider. Then you will have to modify the view in frontend/view/site/index.php to load the pictures you chose. 
If you want to change the menu icons, they are located in frontend/web/images/icons. 
Navigation menu is located in frontend/views/layouts/main.php 
Start page, news section (wiki api) is located in frontend/controllers/SiteController.php 
 
The CSS and JS files are also in the frontend/web folder.

=Adding new search fields or fields to display=
==Adding them in SOLR==
*edit the schema.xml file in SOLR and add the new fields you want to have. If you want to search for them, edit the file for the search core *_search/conf/schema.xml. If wou want to display them in the preview, edit the file for the preview core *_preview/conf/schema.xml. If you need it for the details page, edit the file for the details core *_details/conf/schema.xml. Generally: if you want to search for it (search core), the field has to be indexed, but not mandatory stored. If you want to display it (preview/details), it has to be stored but not indexed. The tripleidstoreid or the occurrenceid *always* has to be indexed.
*restart Tomcat
'''Important ''': always restart Tomcat (service tomcat8 restart) when you modify the schema.xml file.
*edit the Java file corresponding to the core that need to be updated- for example, if you want to index or store the recordbasis, add the following method
<code>
private void addRecordbasis(int datasourceid) throws ClassNotFoundException, SQLException {
System.out.println("RUN RECORDBASIS ");
if(tripleidstoreids.size()>0){
String queryTID =
"SELECT distinct fk_tripleidstoreid as tripleidstoreid, "
+ "IFNULL(recordbasis,'N/A') as recordbasis "
+ " FROM recordbasis "
+ " JOIN occurrence ON recordbasisid = fk_recordbasisid "
+ " WHERE fk_tripleidstoreid in ("+StringUtils.join(tripleidstoreids,", ")+") "
+ " AND fk_datasourceid ="+datasourceid
+ " ORDER BY fk_tripleidstoreid ";
update( queryTID,tripleidstoreids.size());
}
System.gc();
}
</code> and add a call to that method in the '''handleDocuments''' method:
<code>
occurrencesDone.clear();
multivalueForbidden=false;
addRecordbasis(datasourceid );
</code>
*existing documents can not be updated in SOLR in our case (only if everything was "stored"). So first, delete the old data from the core. Depending on how many records there are, you might have to delete the data for a specific repository only solrServer:solrPort/solr/coreName_SEARCH_OR_PREVIEW_OR_DETAILS/update?stream.body=<delete><query>YOUR_LIMITATION_FIELD:*</query></delete>&commit=true This "limitation_field" must be an indexed value in the core. If you don't have a lot of data, you can put <query>*:*</query> as the generation of the new SOLR index will be fast enough (5-10min per Core for 180.000 records).
*per default, it will index this new field for each datasource. If it's still missing at the end for some datasources, or if you only want to do it for specific datasources, edit the '''main''' method <code>for (int i=0;i<166;i++)</code>

==Adding them in the portal==
*Edit the SearchController file (controllers)(additions can be made in the CommonSearchController, deletion only in the specific SearchController if you have several portals running in parallel with the same installation)
**if it's a dropdownlist, add the name of the field in the variable <code>$knownLParams</code>.
**if it's a simple term or a suggestion list, add the name of the field in the variable <code>$knownParams</code>.
*Edit the SearchForm file (models)(same rule as above), edit the variable <code>$_parameters</code> - it can be a text, a select(dropdownlist), a suggest, a radio or a range. Also add the public variable with the same name (ie. <code>public $sampleavailability;</code> and
<code>
'sampleavailability' => [
'type' => 'radio',
'name' => 'Sample availability (loan)',
'config' => [
'size' => 285,
'maxlength' => 150,
'name' => 'sampleavailability'
],
'hideable' => 'true'
]
</code>. It will be automatically displayed in the search form. The '''hideable''' parameter specifies if it is displayed per default or if it is only in the accordion.
Eventually, edit the '''attribteLabels''' function such as <code>'sampleavailability' => 'Sample availability (loan)'</code>
*Edit the listView.php (views) file if it has to be displayed here, or the recordViewTabs.php if it has to be displayed on the details page. Here again, add it to the $suggestFields or $defaultFields or $selectFields or $buttonFields, depending of its type.
*Edit the file CreateFormManager(vendor) only if it's a radio button (look how it's done for cites and sampleavailability).
*Edit the file SOLRQueryManager(vendor) - for example, if it has to look in the SOLR case-unsensitive value, do as for
<code>
case 'fullScientificName' :
return $field . '_nc:' . $value . '';
break;
</code> If it's case sensitive,
<code>
case 'collectioncode' :
return $field . ':"' . $value . '"';
break;
</code>

=Running multiple GPS portals in parallel=
[[File:gps_folders.jpg|thumb|left|300px|Example of two portals running in parallel, specific files are located in "ggbnportal" and "worldfloraonline". All other folders are used by both portal instances]]

*in COMMON_PORTALFILES/common/config/bootstrap.php: add the new portal folder here; required for Yii import routines
*in COMMON_PORTALFILES/environnments/index.php: add the new portal folder here; the name of the folders that will be exposed and therefore where Yii should be authorized to write logs and handle the CSS files
*in the new portal folder: all namespaces must be replaced by the new portal name
*COMMON_PORTALFILES/NEW_Portal/config/params.php and main.php: change database, SOLR and all other parameters

=Debug=
If something does not work as expected, have a look in the catalina.out file (generated by Tomcat) on the machine where SOLR is running. If Tomcat is running through XAMPP, you might have a part of the logs in the catalina.bat window. You will see the last queries sent to SOLR at the end of the file.
The YII file runtime/log/app.log might also help, and is located in the yii folder on the machine running the portal (ie. advanced/runtime....).

Main Page

2016-03-17T07:18:09Z

GabiDroege:

'''Welcome to the Wiki of the GGBN Portal Software'''

=Introduction=
The GGBN Portal Software (GPS) has been developed by the Botanic Garden and Botanical Museum Berlin-Dahlem for the Global Genome Biodiversity Network ([http://www.ggbn.org GGBN]). The project was funded by the German Research Foundation (DFG (01/2014 - 03/2016)). The software is written in php using the [http://www.yiiframework.com/ Yii framework] and fully open source. The GPS allows costumization and therefore can be used for other special interest networks too. It is optimized for an underlying SOLR instance, data are harvested with [http://wiki.bgbm.org/bhit/ B-HIT].

=Workflow=
[[File:Workflow GPS.jpg|500px]]

Data are provided through BioCASe and IPT. The harvester [http://wiki.bgbm.org/bhit B-HIT] stores all original data in a MySQL database. To find out more about B-HIT please check out the [http://wiki.bgbm.org/bhit B-HIT Wiki]. After harvesting data are cleaned and can be matched against different taxonomic backbones. A SOLR instance is created on top of the MySQL database. The GPS itself requires access to the SOLR index as well as the MySQL index.

=Installation=

==You will need==
* Apache (tested with Apache/2.4.7 (Ubuntu), Apache/2.2.22 (Debian), Apache/2.4.10 (Win64)) with libapache2-mod-php5
* PHP5 (from 5.5! tested with 5.5.9, 5.5.30 and 5.5.15). You also might need to install php5-xsl, php5-gd and php5-mysql.
* a SOLR instance (tested with Apache SOLR 4.9.0). Contact us for getting the appropriate SOLR configuration files.
* a MySQL database (tested with MySQL 5.5.46). It will contain data harvested with [http://wiki.bgbm.org/bhit/index.php/Main_Page B-HIT]

Yii (version 2) will be fully contained in the subversion code of the portal software.

==Copy a working version from the svn ==
Create a destination folder, for the rest of this documentation we will choose "my_portal" in your web-folder (ie. /var/www/, or *WINPATH*). 
Get http://ww2.biocase.org/svn/dnabank/Dnabank_Portal/ggbn_portal/trunk/ into "my_portal".

=Configuration=
''TBD''
''yii, Apache, rights, SOLR (SOLR config files also & Java)'' 
First, let say we want to access the portal under the URL http://localhost/ggbn_portal.

*Check that the Apache User has writing/editing rights in the whole "my_portal" folder.
*Edit the file "my_portal/frontend/config/main.php".
**Configure the portal URL: <code>$baseUrl = str_replace ( '/SUBDIR_AFTER_YOUR_WEB_FOLDER/my_portal/frontend/web', '/ggbn_portal', (new Request ())->getBaseUrl () );</code>
**Configure the MySQL database credentials <code>
'db' => [
'class' => '\yii\db\Connection',
'dsn' => 'mysql:host=IP;dbname=DBNAME',
'username' => 'USER',
'password' => 'PASS',
'charset' => 'utf8'
],</code>
*Edit the file "my_portal/frontend/config/params.php".
**e-mails
***Configure the admin e-mail <code>'adminEmail' => 'ADMIN@DOMAIN',</code>. The admin will get error/bug messages.
***Configure the support e-mail(s) <code>'supportEmail' => ['MAIL1','MAIL2'..],</code>.
***Configure the feedback e-mail(s) <code>'feedbackMail' => ['MAIL1','MAIL2'..],</code>. They will get the messages sent with the feedback function.
***Configure the no-reply-mail (who is supposed to sent e-mail(s) to the users) <code>'noreplyMail'=>'MAIL'
**Configure the location of the DarwinCore Archives (harvested with B-HIT) <code>'dwcArchivesPath' => 'PATH_TO_THE_PARENT_FOLDER' ,</code>. Check the access rights! (read is enough), should be an URL if you want to use Annosys
**Configure the location(root) of your SOLR instance <code>'solrhost'=> 'SOLR_HOST'</code> and the SOLR core names
<code>
$searchname="SEARCH_CORE";
$previewname="PREVIEW_CORE";
$detailsname="DETAILS_CORE";
</code>
There is also a core for the statistics pages (<code>ggbn_stats</code>).
**Set the temporary folder <code>'tmpFolder'=>'TEMP_FOLDER'</code>. Check the access rights! (write is needed)
**Configure the options
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'noBack'=>true_OR_false</code>: a special backbone is used
***<code>'shopping'=>true_OR_false</code>: the shopping system is enabled or disabled
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'newsService'=>true_OR_false</code>: uses the GGBN news services - if it has to be used with a different wiki news, edit the SiteController.py
***<code>'viewsCountsService'=>true_OR_false</code>: uses the GGBN specific views containing some counts (number of species, dna, ...). Look in the '''GGBN specific configuration''' for the SQL statements.
***<code>'displayMore'=>true_OR_false</code>: displays the categories with internal links on the details page
***<code>'displayExternals'=>true_OR_false</code>: displays the categories with external links (GBIF, BOLD, NCBI...) on the details page
***<code>'ABCDdownloable'=>true_OR_false</code>: ABCD files are downloadable or not
***<code>'useAnnosys'=>true_OR_false</code>: link to annosys (annotation system) on the details page
***<code>'annosysAP'=>'https://annosys.bgbm.fu-berlin.de/AnnoSys/AnnoSys?providerURL='</code>: root URL for Annosys
***<code>'siteName'=>'my_portal',</code>: the portal name (used in the URL)
***<code>'siteTitle'=>'GGBN Portal',</code>: the default site name/title
***<code>'catalogNumberClickable'=>true,</code>: if the catalogNumber/UnitID can be clickable on the details page

*Edit the Apache configuration file, for example:
<pre>
Alias /ggbn_portal "/var/www/my_portal/frontend/web"
<Directory "/var/www/my_portal/frontend/web">
Options All
AllowOverride All
</Directory>
</pre>

=Personalisation=
If you want to change the icon (favicon), it is located in frontend/web. 
If you want to change the slider images on the home page, they are located in frontend/web/images/slider. Then you will have to modify the view in frontend/view/site/index.php to load the pictures you chose. 
If you want to change the menu icons, they are located in frontend/web/images/icons. 
Navigation menu is located in frontend/views/layouts/main.php 
Start page, news section (wiki api) is located in frontend/controllers/SiteController.php 
 
The CSS and JS files are also in the frontend/web folder.

=Adding new search fields or fields to display=
==Adding them in SOLR==
*edit the schema.xml file in SOLR and add the new fields you want to have. If you want to search for them, edit the file for the search core *_search/conf/schema.xml. If wou want to display them in the preview, edit the file for the preview core *_preview/conf/schema.xml. If you need it for the details page, edit the file for the details core *_details/conf/schema.xml. Generally: if you want to search for it (search core), the field has to be indexed, but not mandatory stored. If you want to display it (preview/details), it has to be stored but not indexed. The tripleidstoreid or the occurrenceid *always* has to be indexed.
*restart Tomcat
'''Important ''': always restart Tomcat (service tomcat8 restart) when you modify the schema.xml file.
*edit the Java file corresponding to the core that need to be updated- for example, if you want to index or store the recordbasis, add the following method
<code>
private void addRecordbasis(int datasourceid) throws ClassNotFoundException, SQLException {
System.out.println("RUN RECORDBASIS ");
if(tripleidstoreids.size()>0){
String queryTID =
"SELECT distinct fk_tripleidstoreid as tripleidstoreid, "
+ "IFNULL(recordbasis,'N/A') as recordbasis "
+ " FROM recordbasis "
+ " JOIN occurrence ON recordbasisid = fk_recordbasisid "
+ " WHERE fk_tripleidstoreid in ("+StringUtils.join(tripleidstoreids,", ")+") "
+ " AND fk_datasourceid ="+datasourceid
+ " ORDER BY fk_tripleidstoreid ";
update( queryTID,tripleidstoreids.size());
}
System.gc();
}
</code> and add a call to that method in the '''handleDocuments''' method:
<code>
occurrencesDone.clear();
multivalueForbidden=false;
addRecordbasis(datasourceid );
</code>
*existing documents can not be updated in SOLR in our case (only if everything was "stored"). So first, delete the old data from the core. Depending on how many records there are, you might have to delete the data for a specific repository only solrServer:solrPort/solr/coreName_SEARCH_OR_PREVIEW_OR_DETAILS/update?stream.body=<delete><query>YOUR_LIMITATION_FIELD:*</query></delete>&commit=true This "limitation_field" must be an indexed value in the core. If you don't have a lot of data, you can put <query>*:*</query> as the generation of the new SOLR index will be fast enough (5-10min per Core for 180.000 records).
*per default, it will index this new field for each datasource. If it's still missing at the end for some datasources, or if you only want to do it for specific datasources, edit the '''main''' method <code>for (int i=0;i<166;i++)</code>

==Adding them in the portal==
*Edit the SearchController file (controllers)(additions can be made in the CommonSearchController, deletion only in the specific SearchController if you have several portals running in parallel with the same installation)
**if it's a dropdownlist, add the name of the field in the variable <code>$knownLParams</code>.
**if it's a simple term or a suggestion list, add the name of the field in the variable <code>$knownParams</code>.
*Edit the SearchForm file (models)(same rule as above), edit the variable <code>$_parameters</code> - it can be a text, a select(dropdownlist), a suggest, a radio or a range. Also add the public variable with the same name (ie. <code>public $sampleavailability;</code> and
<code>
'sampleavailability' => [
'type' => 'radio',
'name' => 'Sample availability (loan)',
'config' => [
'size' => 285,
'maxlength' => 150,
'name' => 'sampleavailability'
],
'hideable' => 'true'
]
</code>. It will be automatically displayed in the search form. The '''hideable''' parameter specifies if it is displayed per default or if it is only in the accordion.
Eventually, edit the '''attribteLabels''' function such as <code>'sampleavailability' => 'Sample availability (loan)'</code>
*Edit the listView.php (views) file if it has to be displayed here, or the recordViewTabs.php if it has to be displayed on the details page. Here again, add it to the $suggestFields or $defaultFields or $selectFields or $buttonFields, depending of its type.
*Edit the file CreateFormManager(vendor) only if it's a radio button (look how it's done for cites and sampleavailability).
*Edit the file SOLRQueryManager(vendor) - for example, if it has to look in the SOLR case-unsensitive value, do as for
<code>
case 'fullScientificName' :
return $field . '_nc:' . $value . '';
break;
</code> If it's case sensitive,
<code>
case 'collectioncode' :
return $field . ':"' . $value . '"';
break;
</code>

=Running multiple GPS portals in parallel=
[[File:gps_folders.jpg|thumb|left|300px|Example of two portals running in parallel, specific files are located in "ggbnportal" and "worldfloraonline". All other folders are used by both portal instances]]

*in common/config/bootstrap: the name of all folders that Yii2 should go through (for imports and so)
*in environnments/index.php: the name of the folders that will be exposed and therefore where Yii should be authorized to write logs and handle the CSS files

=Debug=
If something does not work as expected, have a look in the catalina.out file (generated by Tomcat) on the machine where SOLR is running. If Tomcat is running through XAMPP, you might have a part of the logs in the catalina.bat window. You will see the last queries sent to SOLR at the end of the file.
The YII file runtime/log/app.log might also help, and is located in the yii folder on the machine running the portal (ie. advanced/runtime....).

Main Page

2016-03-17T07:11:30Z

GabiDroege: /* Personalisation */

'''Welcome to the Wiki of the GGBN Portal Software'''

Further information about the software will follow soon.

=Introduction=
The GGBN Portal Software (GPS) has been developed by the Botanic Garden and Botanical Museum Berlin-Dahlem for the Global Genome Biodiversity Network ([http://www.ggbn.org GGBN]). The project was funded by the German Research Foundation (DFG (01/2014 - 03/2016)). The software is written in php using the [http://www.yiiframework.com/ Yii framework] and fully open source. The GPS allows costumization and therefore can be used for other special interest networks too. It is optimized for an underlying SOLR instance, data are harvested with [http://wiki.bgbm.org/bhit/ B-HIT].

=Workflow=
[[File:Workflow GPS.jpg|500px]]

Data are provided through BioCASe and IPT. The harvester [http://wiki.bgbm.org/bhit B-HIT] stores all original data in a MySQL database. To find out more about B-HIT please check out the [http://wiki.bgbm.org/bhit B-HIT Wiki]. After harvesting data are cleaned and can be matched against different taxonomic backbones. A SOLR instance is created on top of the MySQL database. The GPS itself requires access to the SOLR index as well as the MySQL index.

=Installation=

==You will need==
* Apache (tested with Apache/2.4.7 (Ubuntu), Apache/2.2.22 (Debian), Apache/2.4.10 (Win64)) with libapache2-mod-php5
* PHP5 (from 5.5! tested with 5.5.9, 5.5.30 and 5.5.15). You also might need to install php5-xsl, php5-gd and php5-mysql.
* a SOLR instance (tested with Apache SOLR 4.9.0). Contact us for getting the appropriate SOLR configuration files.
* a MySQL database (tested with MySQL 5.5.46). It will contain data harvested with [http://wiki.bgbm.org/bhit/index.php/Main_Page B-HIT]

Yii (version 2) will be fully contained in the subversion code of the portal software.

==Copy a working version from the svn ==
Create a destination folder, for the rest of this documentation we will choose "my_portal" in your web-folder (ie. /var/www/, or *WINPATH*). 
Get http://ww2.biocase.org/svn/dnabank/Dnabank_Portal/ggbn_portal/trunk/ into "my_portal".

=Configuration=
''TBD''
''yii, Apache, rights, SOLR (SOLR config files also & Java)'' 
First, let say we want to access the portal under the URL http://localhost/ggbn_portal.

*Check that the Apache User has writing/editing rights in the whole "my_portal" folder.
*Edit the file "my_portal/frontend/config/main.php".
**Configure the portal URL: <code>$baseUrl = str_replace ( '/SUBDIR_AFTER_YOUR_WEB_FOLDER/my_portal/frontend/web', '/ggbn_portal', (new Request ())->getBaseUrl () );</code>
**Configure the MySQL database credentials <code>
'db' => [
'class' => '\yii\db\Connection',
'dsn' => 'mysql:host=IP;dbname=DBNAME',
'username' => 'USER',
'password' => 'PASS',
'charset' => 'utf8'
],</code>
*Edit the file "my_portal/frontend/config/params.php".
**e-mails
***Configure the admin e-mail <code>'adminEmail' => 'ADMIN@DOMAIN',</code>. The admin will get error/bug messages.
***Configure the support e-mail(s) <code>'supportEmail' => ['MAIL1','MAIL2'..],</code>.
***Configure the feedback e-mail(s) <code>'feedbackMail' => ['MAIL1','MAIL2'..],</code>. They will get the messages sent with the feedback function.
***Configure the no-reply-mail (who is supposed to sent e-mail(s) to the users) <code>'noreplyMail'=>'MAIL'
**Configure the location of the DarwinCore Archives (harvested with B-HIT) <code>'dwcArchivesPath' => 'PATH_TO_THE_PARENT_FOLDER' ,</code>. Check the access rights! (read is enough), should be an URL if you want to use Annosys
**Configure the location(root) of your SOLR instance <code>'solrhost'=> 'SOLR_HOST'</code> and the SOLR core names
<code>
$searchname="SEARCH_CORE";
$previewname="PREVIEW_CORE";
$detailsname="DETAILS_CORE";
</code>
There is also a core for the statistics pages (<code>ggbn_stats</code>).
**Set the temporary folder <code>'tmpFolder'=>'TEMP_FOLDER'</code>. Check the access rights! (write is needed)
**Configure the options
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'noBack'=>true_OR_false</code>: a special backbone is used
***<code>'shopping'=>true_OR_false</code>: the shopping system is enabled or disabled
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'newsService'=>true_OR_false</code>: uses the GGBN news services - if it has to be used with a different wiki news, edit the SiteController.py
***<code>'viewsCountsService'=>true_OR_false</code>: uses the GGBN specific views containing some counts (number of species, dna, ...). Look in the '''GGBN specific configuration''' for the SQL statements.
***<code>'displayMore'=>true_OR_false</code>: displays the categories with internal links on the details page
***<code>'displayExternals'=>true_OR_false</code>: displays the categories with external links (GBIF, BOLD, NCBI...) on the details page
***<code>'ABCDdownloable'=>true_OR_false</code>: ABCD files are downloadable or not
***<code>'useAnnosys'=>true_OR_false</code>: link to annosys (annotation system) on the details page
***<code>'annosysAP'=>'https://annosys.bgbm.fu-berlin.de/AnnoSys/AnnoSys?providerURL='</code>: root URL for Annosys
***<code>'siteName'=>'my_portal',</code>: the portal name (used in the URL)
***<code>'siteTitle'=>'GGBN Portal',</code>: the default site name/title
***<code>'catalogNumberClickable'=>true,</code>: if the catalogNumber/UnitID can be clickable on the details page

*Edit the Apache configuration file, for example:
<pre>
Alias /ggbn_portal "/var/www/my_portal/frontend/web"
<Directory "/var/www/my_portal/frontend/web">
Options All
AllowOverride All
</Directory>
</pre>

=Personalisation=
If you want to change the icon (favicon), it is located in frontend/web. 
If you want to change the slider images on the home page, they are located in frontend/web/images/slider. Then you will have to modify the view in frontend/view/site/index.php to load the pictures you chose. 
If you want to change the menu icons, they are located in frontend/web/images/icons. 
Navigation menu is located in frontend/views/layouts/main.php 
Start page, news section (wiki api) is located in frontend/controllers/SiteController.php 
 
The CSS and JS files are also in the frontend/web folder.

=Adding new search fields or fields to display=
==Adding them in SOLR==
*edit the schema.xml file in SOLR and add the new fields you want to have. If you want to search for them, edit the file for the search core *_search/conf/schema.xml. If wou want to display them in the preview, edit the file for the preview core *_preview/conf/schema.xml. If you need it for the details page, edit the file for the details core *_details/conf/schema.xml. Generally: if you want to search for it (search core), the field has to be indexed, but not mandatory stored. If you want to display it (preview/details), it has to be stored but not indexed. The tripleidstoreid or the occurrenceid *always* has to be indexed.
*restart Tomcat
'''Important ''': always restart Tomcat (service tomcat8 restart) when you modify the schema.xml file.
*edit the Java file corresponding to the core that need to be updated- for example, if you want to index or store the recordbasis, add the following method
<code>
private void addRecordbasis(int datasourceid) throws ClassNotFoundException, SQLException {
System.out.println("RUN RECORDBASIS ");
if(tripleidstoreids.size()>0){
String queryTID =
"SELECT distinct fk_tripleidstoreid as tripleidstoreid, "
+ "IFNULL(recordbasis,'N/A') as recordbasis "
+ " FROM recordbasis "
+ " JOIN occurrence ON recordbasisid = fk_recordbasisid "
+ " WHERE fk_tripleidstoreid in ("+StringUtils.join(tripleidstoreids,", ")+") "
+ " AND fk_datasourceid ="+datasourceid
+ " ORDER BY fk_tripleidstoreid ";
update( queryTID,tripleidstoreids.size());
}
System.gc();
}
</code> and add a call to that method in the '''handleDocuments''' method:
<code>
occurrencesDone.clear();
multivalueForbidden=false;
addRecordbasis(datasourceid );
</code>
*existing documents can not be updated in SOLR in our case (only if everything was "stored"). So first, delete the old data from the core. Depending on how many records there are, you might have to delete the data for a specific repository only solrServer:solrPort/solr/coreName_SEARCH_OR_PREVIEW_OR_DETAILS/update?stream.body=<delete><query>YOUR_LIMITATION_FIELD:*</query></delete>&commit=true This "limitation_field" must be an indexed value in the core. If you don't have a lot of data, you can put <query>*:*</query> as the generation of the new SOLR index will be fast enough (5-10min per Core for 180.000 records).
*per default, it will index this new field for each datasource. If it's still missing at the end for some datasources, or if you only want to do it for specific datasources, edit the '''main''' method <code>for (int i=0;i<166;i++)</code>

==Adding them in the portal==
*Edit the SearchController file (controllers)(additions can be made in the CommonSearchController, deletion only in the specific SearchController if you have several portals running in parallel with the same installation)
**if it's a dropdownlist, add the name of the field in the variable <code>$knownLParams</code>.
**if it's a simple term or a suggestion list, add the name of the field in the variable <code>$knownParams</code>.
*Edit the SearchForm file (models)(same rule as above), edit the variable <code>$_parameters</code> - it can be a text, a select(dropdownlist), a suggest, a radio or a range. Also add the public variable with the same name (ie. <code>public $sampleavailability;</code> and
<code>
'sampleavailability' => [
'type' => 'radio',
'name' => 'Sample availability (loan)',
'config' => [
'size' => 285,
'maxlength' => 150,
'name' => 'sampleavailability'
],
'hideable' => 'true'
]
</code>. It will be automatically displayed in the search form. The '''hideable''' parameter specifies if it is displayed per default or if it is only in the accordion.
Eventually, edit the '''attribteLabels''' function such as <code>'sampleavailability' => 'Sample availability (loan)'</code>
*Edit the listView.php (views) file if it has to be displayed here, or the recordViewTabs.php if it has to be displayed on the details page. Here again, add it to the $suggestFields or $defaultFields or $selectFields or $buttonFields, depending of its type.
*Edit the file CreateFormManager(vendor) only if it's a radio button (look how it's done for cites and sampleavailability).
*Edit the file SOLRQueryManager(vendor) - for example, if it has to look in the SOLR case-unsensitive value, do as for
<code>
case 'fullScientificName' :
return $field . '_nc:' . $value . '';
break;
</code> If it's case sensitive,
<code>
case 'collectioncode' :
return $field . ':"' . $value . '"';
break;
</code>

=Running multiple GPS portals in parallel=
[[File:gps_folders.jpg|thumb|left|300px|Example of two portals running in parallel, specific files are located in "ggbnportal" and "worldfloraonline". All other folders are used by both portal instances]]

*in common/config/bootstrap: the name of all folders that Yii2 should go through (for imports and so)
*in environnments/index.php: the name of the folders that will be exposed and therefore where Yii should be authorized to write logs and handle the CSS files

=Debug=
If something does not work as expected, have a look in the catalina.out file (generated by Tomcat) on the machine where SOLR is running. If Tomcat is running through XAMPP, you might have a part of the logs in the catalina.bat window. You will see the last queries sent to SOLR at the end of the file.
The YII file runtime/log/app.log might also help, and is located in the yii folder on the machine running the portal (ie. advanced/runtime....).

MediaWiki:Sidebar

2016-02-09T16:12:18Z

GabiDroege:

* SEARCH
*Home
**Main_Page | GPS Wiki

*Resources
**GGBN_specific_configuration | GGBN specific configuration
**http://wiki.bgbm.org/bps/ | B-HIT Wiki

*Instances
**http://www.ggbn.org | GGBN
**http://wfospecimens.cybertaxonomy.org/ | World Flora Online Specimens

* TOOLBOX

GGBN specific configuration

2016-02-09T16:10:12Z

GabiDroege: GabiDroege moved page GGBN specific- configuration to GGBN specific configuration without leaving a redirect

* Several views are required for the counts-box on start page.
<pre>
CREATE
ALGORITHM = UNDEFINED
DEFINER = `root`@`localhost`
SQL SECURITY DEFINER
VIEW `counts` AS
select
count(`ro`.`occurrenceid`) AS `counts`, 'DNA' AS `kind`
from
(`ggbn_index`.`rawoccurrence` `ro`
join `ggbn_index`.`unitkind` `uk` ON ((`ro`.`fk_kindofunitid` = `uk`.`unitkindid`)))
where
(`uk`.`kindofunit_clean` = 'DNA')
union select
count(`ro`.`occurrenceid`) AS `counts`, 'Tissues' AS `kind`
from
(`ggbn_index`.`rawoccurrence` `ro`
join `ggbn_index`.`unitkind` `uk` ON ((`ro`.`fk_kindofunitid` = `uk`.`unitkindid`)))
where
(`uk`.`kindofunit_clean` = 'tissue')
union select
count(`ro`.`occurrenceid`) AS `counts`, 'Cultures' AS `kind`
from
(`ggbn_index`.`rawoccurrence` `ro`
join `ggbn_index`.`unitkind` `uk` ON ((`ro`.`fk_kindofunitid` = `uk`.`unitkindid`)))
where
(`uk`.`kindofunit_clean` = 'Culture')
union select
count(`ro`.`occurrenceid`) AS `counts`,
'eVouchers' AS `kind`
from
(`ggbn_index`.`rawoccurrence` `ro`
join `ggbn_index`.`unitkind` `uk` ON ((`ro`.`fk_kindofunitid` = `uk`.`unitkindid`)))
where
(`uk`.`kindofunit_clean` = 'eVoucher')
union select
count(`ro`.`occurrenceid`) AS `counts`,
'Specimens' AS `kind`
from
(`ggbn_index`.`rawoccurrence` `ro`
join `ggbn_index`.`unitkind` `uk` ON ((`ro`.`fk_kindofunitid` = `uk`.`unitkindid`)))
where
(`uk`.`kindofunit_clean` = 'specimen')
union select
count(`ro`.`occurrenceid`) AS `counts`, 'Unknown' AS `kind`
from
(`ggbn_index`.`rawoccurrence` `ro`
join `ggbn_index`.`unitkind` `uk` ON ((`ro`.`fk_kindofunitid` = `uk`.`unitkindid`)))
where
(`uk`.`kindofunit_clean` = 'unknown')
union select
count(`ro`.`occurrenceid`) AS `counts`, 'Enviros' AS `kind`
from
(`ggbn_index`.`rawoccurrence` `ro`
join `ggbn_index`.`unitkind` `uk` ON ((`ro`.`fk_kindofunitid` = `uk`.`unitkindid`)))
where
(`uk`.`kindofunit_clean` = 'environmental sample')
union select
count(`grouped_fullscientificname`.`fullscientificname`) AS `counts`,
'Taxa' AS `kind`
from
`ggbn_index`.`grouped_fullscientificname`
union select
count(distinct `families_view`.`family`) AS `counts`,
'Families' AS `kind`
from
`ggbn_index`.`families_view`
union select
count(distinct `genera_view`.`genus`) AS `counts`,
'Genera' AS `kind`
from
`ggbn_index`.`genera_view`
union select
count(distinct `species_view`.`species`) AS `counts`,
'Species' AS `kind`
from
`ggbn_index`.`species_view`
</pre>
<pre>
CREATE
ALGORITHM = UNDEFINED
DEFINER = `root`@`localhost`
SQL SECURITY DEFINER
VIEW `families_view` AS
select distinct
`f`.`family` AS `family`
from
((`backbone`.`family` `f`
join `backbone`.`name` `n` ON ((`f`.`familykey` = `n`.`familyKey`)))
join `ggbn_index`.`identification` `i` ON ((`i`.`gbifKey` = `n`.`acceptedKey`)))
union select distinct
`f`.`family` AS `family`
from
((`backbone_col`.`family` `f`
join `backbone_col`.`name` `n` ON ((`f`.`familykey` = `n`.`familyKey`)))
join `ggbn_index`.`identification` `i` ON ((`i`.`colKey` = `n`.`acceptedKey`)))
union select distinct
`f`.`family` AS `family`
from
((`backbone_ncbi`.`family` `f`
join `backbone_ncbi`.`name` `n` ON ((`f`.`familykey` = `n`.`familyKey`)))
join `ggbn_index`.`identification` `i` ON ((`i`.`ncbiKey` = `n`.`acceptedKey`)))
</pre>
<pre>
CREATE
ALGORITHM = UNDEFINED
DEFINER = `root`@`localhost`
SQL SECURITY DEFINER
VIEW `genera_view` AS
select distinct
`f`.`genus` AS `genus`
from
((`backbone`.`genus` `f`
join `backbone`.`name` `n` ON ((`f`.`genuskey` = `n`.`genusKey`)))
join `ggbn_index`.`identification` `i` ON ((`i`.`gbifKey` = `n`.`acceptedKey`)))
union select distinct
`f`.`genus` AS `genus`
from
((`backbone_col`.`genus` `f`
join `backbone_col`.`name` `n` ON ((`f`.`genuskey` = `n`.`genusKey`)))
join `ggbn_index`.`identification` `i` ON ((`i`.`colKey` = `n`.`acceptedKey`)))
union select distinct
`f`.`genus` AS `genus`
from
((`backbone_ncbi`.`genus` `f`
join `backbone_ncbi`.`name` `n` ON ((`f`.`genuskey` = `n`.`genusKey`)))
join `ggbn_index`.`identification` `i` ON ((`i`.`ncbiKey` = `n`.`acceptedKey`)))
</pre>
<pre>
CREATE
ALGORITHM = UNDEFINED
DEFINER = `root`@`localhost`
SQL SECURITY DEFINER
VIEW `grouped_fullscientificname` AS
select
`identification`.`fullScientificName` AS `fullscientificname`
from
`identification`
group by `identification`.`fullScientificName`
</pre>
<pre>
CREATE
ALGORITHM = UNDEFINED
DEFINER = `root`@`localhost`
SQL SECURITY DEFINER
VIEW `species_view` AS
select distinct
`n2`.`canonicalName` AS `species`
from
((`backbone`.`name` `n`
join `backbone`.`name` `n2` ON ((`n`.`acceptedKey` = `n2`.`namekey`)))
join `ggbn_index`.`identification` `i` ON ((`i`.`gbifKey` = `n`.`acceptedKey`)))
where
((`n`.`rank` = 'SPECIES')
and (`n2`.`rank` = 'SPECIES'))
union select distinct
`n2`.`canonicalName` AS `species`
from
((`backbone_col`.`name` `n`
join `backbone_col`.`name` `n2` ON ((`n`.`acceptedKey` = `n2`.`namekey`)))
join `ggbn_index`.`identification` `i` ON ((`i`.`colKey` = `n`.`acceptedKey`)))
where
((`n`.`rank` = 'SPECIES')
and (`n2`.`rank` = 'SPECIES'))
union select distinct
`n2`.`canonicalName` AS `species`
from
((`backbone_ncbi`.`name` `n`
join `backbone_ncbi`.`name` `n2` ON ((`n`.`acceptedKey` = `n2`.`namekey`)))
join `ggbn_index`.`identification` `i` ON ((`i`.`ncbiKey` = `n`.`acceptedKey`)))
where
((`n`.`rank` = 'SPECIES')
and (`n2`.`rank` = 'SPECIES'))
</pre>
* The institution has to be filled in bio_datasource (city and institution, for the harvester_factoy like '%Harvester%') - but first the datasource has to be added (metadata update). It will be used for the repository/registry field in the GGBN portal.
* The parentInstitution table is used for the contacts and must be filled manually.
* The GGBN registry is based on the NCD software and requires an additional database connection in the config file. Contact us for more information.
* For user management (login/shopping), a few extra tables are required. Follow the instructions from the file common/controllers/CommonPermissionController.php to create and populate them if you do not already have them from the SQL-Dump. You will also need an extra view for the statistics:
<pre>
CREATE
ALGORITHM = UNDEFINED
DEFINER = `root`@`localhost`
SQL SECURITY DEFINER
VIEW `shopping_view` AS
select distinct
`oc`.`fk_tripleidstoreid` AS `tripleidstoreid`,
`bs`.`id` AS `data_source_id`,
`pit`.`parentInstitutionID` AS `parentInstitutionID`,
`pit`.`institutionShort` AS `institutionShort`,
`pit`.`logoURL` AS `logoURL`
from
((`bio_datasource` `bs`
join `occurrence` `oc` ON ((`bs`.`id` = `oc`.`fk_datasourceid`)))
join `parentInstitution` `pit` ON ((`bs`.`fk_parentInstitutionid` = `pit`.`parentInstitutionID`)))
</pre>

Main Page

2016-02-09T16:07:16Z

GabiDroege: /* GGBN specific configuration */

'''Welcome to the Wiki of the GGBN Portal Software'''

Further information about the software will follow soon.

=Introduction=
The GGBN Portal Software (GPS) has been developed by the Botanic Garden and Botanical Museum Berlin-Dahlem for the Global Genome Biodiversity Network ([http://www.ggbn.org GGBN]). The project was funded by the German Research Foundation (DFG (01/2014 - 03/2016)). The software is written in php using the [http://www.yiiframework.com/ Yii framework] and fully open source. The GPS allows costumization and therefore can be used for other special interest networks too. It is optimized for an underlying SOLR instance, data are harvested with [http://wiki.bgbm.org/bhit/ B-HIT].

=Workflow=
[[File:Workflow GPS.jpg|500px]]

Data are provided through BioCASe and IPT. The harvester [http://wiki.bgbm.org/bhit B-HIT] stores all original data in a MySQL database. To find out more about B-HIT please check out the [http://wiki.bgbm.org/bhit B-HIT Wiki]. After harvesting data are cleaned and can be matched against different taxonomic backbones. A SOLR instance is created on top of the MySQL database. The GPS itself requires access to the SOLR index as well as the MySQL index.

=Installation=

==You will need==
* Apache (tested with Apache/2.4.7 (Ubuntu), Apache/2.2.22 (Debian), Apache/2.4.10 (Win64)) with libapache2-mod-php5
* PHP5 (from 5.5! tested with 5.5.9, 5.5.30 and 5.5.15). You also might need to install php5-xsl, php5-gd and php5-mysql.
* a SOLR instance (tested with Apache SOLR 4.9.0). Contact us for getting the appropriate SOLR configuration files.
* a MySQL database (tested with MySQL 5.5.46). It will contain data harvested with [http://wiki.bgbm.org/bhit/index.php/Main_Page B-HIT]

Yii (version 2) will be fully contained in the subversion code of the portal software.

==Copy a working version from the svn ==
Create a destination folder, for the rest of this documentation we will choose "my_portal" in your web-folder (ie. /var/www/, or *WINPATH*). 
Get http://ww2.biocase.org/svn/dnabank/Dnabank_Portal/ggbn_portal/trunk/ into "my_portal".

=Configuration=
''TBD''
''yii, Apache, rights, SOLR (SOLR config files also & Java)'' 
First, let say we want to access the portal under the URL http://localhost/ggbn_portal.

*Check that the Apache User has writing/editing rights in the whole "my_portal" folder.
*Edit the file "my_portal/frontend/config/main.php".
**Configure the portal URL: <code>$baseUrl = str_replace ( '/SUBDIR_AFTER_YOUR_WEB_FOLDER/my_portal/frontend/web', '/ggbn_portal', (new Request ())->getBaseUrl () );</code>
**Configure the MySQL database credentials <code>
'db' => [
'class' => '\yii\db\Connection',
'dsn' => 'mysql:host=IP;dbname=DBNAME',
'username' => 'USER',
'password' => 'PASS',
'charset' => 'utf8'
],</code>
*Edit the file "my_portal/frontend/config/params.php".
**e-mails
***Configure the admin e-mail <code>'adminEmail' => 'ADMIN@DOMAIN',</code>. The admin will get error/bug messages.
***Configure the support e-mail(s) <code>'supportEmail' => ['MAIL1','MAIL2'..],</code>.
***Configure the feedback e-mail(s) <code>'feedbackMail' => ['MAIL1','MAIL2'..],</code>. They will get the messages sent with the feedback function.
***Configure the no-reply-mail (who is supposed to sent e-mail(s) to the users) <code>'noreplyMail'=>'MAIL'
**Configure the location of the DarwinCore Archives (harvested with B-HIT) <code>'dwcArchivesPath' => 'PATH_TO_THE_PARENT_FOLDER' ,</code>. Check the access rights! (read is enough)
**Configure the location(root) of your SOLR instance <code>'solrhost'=> 'SOLR_HOST'</code> and the SOLR core names
<code>
$searchname="SEARCH_CORE";
$previewname="PREVIEW_CORE";
$detailsname="DETAILS_CORE";
</code>
There is also a core for the statistics pages (<code>ggbn_stats</code>).
**Set the temporary folder <code>'tmpFolder'=>'TEMP_FOLDER'</code>. Check the access rights! (write is needed)
**Configure the options
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'noBack'=>true_OR_false</code>: a special backbone is used
***<code>'shopping'=>true_OR_false</code>: the shopping system is enabled or disabled
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'newsService'=>true_OR_false</code>: uses the GGBN news services - if it has to be used with a different wiki news, edit the SiteController.py
***<code>'viewsCountsService'=>true_OR_false</code>: uses the GGBN specific views containing some counts (number of species, dna, ...). Look in the '''GGBN specific configuration''' for the SQL statements.
***<code>'displayMore'=>true_OR_false</code>: displays the categories with internal links on the details page
***<code>'displayExternals'=>true_OR_false</code>: displays the categories with external links (GBIF, BOLD, NCBI...) on the details page
***<code>'ABCDdownloable'=>true_OR_false</code>: ABCD files are downloadable or not
***<code>'useAnnosys'=>true_OR_false</code>: link to annosys (annotation system) on the details page
***<code>'annosysAP'=>'https://annosys.bgbm.fu-berlin.de/AnnoSys/AnnoSys?providerURL='</code>: root URL for Annosys
***<code>'siteName'=>'my_portal',</code>: the portal name (used in the URL)
***<code>'siteTitle'=>'GGBN Portal',</code>: the default site name/title
***<code>'catalogNumberClickable'=>true,</code>: if the catalogNumber/UnitID can be clickable on the details page

*Edit the Apache configuration file, for example:
<pre>
Alias /ggbn_portal "/var/www/my_portal/frontend/web"
<Directory "/var/www/my_portal/frontend/web">
Options All
AllowOverride All
</Directory>
</pre>

=Personalisation=
If you want to change the icon (favicon), it is located in frontend/web. 
If you want to change the slider images on the home page, they are located in frontend/web/images/slider. Then you will have to modify the view in frontend/view/site/index.php to load the pictures you chose. 
If you want to change the menu icons, they are located in frontend/web/images/icons. 
 
The CSS and JS files are also in the frontend/web folder.

=Adding new search fields or fields to display=
==Adding them in SOLR==
*edit the schema.xml file in SOLR and add the new fields you want to have. If you want to search for them, edit the file for the search core *_search/conf/schema.xml. If wou want to display them in the preview, edit the file for the preview core *_preview/conf/schema.xml. If you need it for the details page, edit the file for the details core *_details/conf/schema.xml. Generally: if you want to search for it (search core), the field has to be indexed, but not mandatory stored. If you want to display it (preview/details), it has to be stored but not indexed. The tripleidstoreid or the occurrenceid *always* has to be indexed.
*restart Tomcat
'''Important ''': always restart Tomcat (service tomcat8 restart) when you modify the schema.xml file.
*edit the Java file corresponding to the core that need to be updated- for example, if you want to index or store the recordbasis, add the following method
<code>
private void addRecordbasis(int datasourceid) throws ClassNotFoundException, SQLException {
System.out.println("RUN RECORDBASIS ");
if(tripleidstoreids.size()>0){
String queryTID =
"SELECT distinct fk_tripleidstoreid as tripleidstoreid, "
+ "IFNULL(recordbasis,'N/A') as recordbasis "
+ " FROM recordbasis "
+ " JOIN occurrence ON recordbasisid = fk_recordbasisid "
+ " WHERE fk_tripleidstoreid in ("+StringUtils.join(tripleidstoreids,", ")+") "
+ " AND fk_datasourceid ="+datasourceid
+ " ORDER BY fk_tripleidstoreid ";
update( queryTID,tripleidstoreids.size());
}
System.gc();
}
</code> and add a call to that method in the '''handleDocuments''' method:
<code>
occurrencesDone.clear();
multivalueForbidden=false;
addRecordbasis(datasourceid );
</code>
*existing documents can not be updated in SOLR in our case (only if everything was "stored"). So first, delete the old data from the core. Depending on how many records there are, you might have to delete the data for a specific repository only solrServer:solrPort/solr/coreName_SEARCH_OR_PREVIEW_OR_DETAILS/update?stream.body=<delete><query>YOUR_LIMITATION_FIELD:*</query></delete>&commit=true This "limitation_field" must be an indexed value in the core. If you don't have a lot of data, you can put <query>*:*</query> as the generation of the new SOLR index will be fast enough (5-10min per Core for 180.000 records).
*per default, it will index this new field for each datasource. If it's still missing at the end for some datasources, or if you only want to do it for specific datasources, edit the '''main''' method <code>for (int i=0;i<166;i++)</code>

==Adding them in the portal==
*Edit the SearchController file (controllers)(additions can be made in the CommonSearchController, deletion only in the specific SearchController if you have several portals running in parallel with the same installation)
**if it's a dropdownlist, add the name of the field in the variable <code>$knownLParams</code>.
**if it's a simple term or a suggestion list, add the name of the field in the variable <code>$knownParams</code>.
*Edit the SearchForm file (models)(same rule as above), edit the variable <code>$_parameters</code> - it can be a text, a select(dropdownlist), a suggest, a radio or a range. Also add the public variable with the same name (ie. <code>public $sampleavailability;</code> and
<code>
'sampleavailability' => [
'type' => 'radio',
'name' => 'Sample availability (loan)',
'config' => [
'size' => 285,
'maxlength' => 150,
'name' => 'sampleavailability'
],
'hideable' => 'true'
]
</code>. It will be automatically displayed in the search form. The '''hideable''' parameter specifies if it is displayed per default or if it is only in the accordion.
Eventually, edit the '''attribteLabels''' function such as <code>'sampleavailability' => 'Sample availability (loan)'</code>
*Edit the listView.php (views) file if it has to be displayed here, or the recordViewTabs.php if it has to be displayed on the details page. Here again, add it to the $suggestFields or $defaultFields or $selectFields or $buttonFields, depending of its type.
*Edit the file CreateFormManager(vendor) only if it's a radio button (look how it's done for cites and sampleavailability).
*Edit the file SOLRQueryManager(vendor) - for example, if it has to look in the SOLR case-unsensitive value, do as for
<code>
case 'fullScientificName' :
return $field . '_nc:' . $value . '';
break;
</code> If it's case sensitive,
<code>
case 'collectioncode' :
return $field . ':"' . $value . '"';
break;
</code>

=Running multiple GPS portals in parallel=
[[File:gps_folders.jpg|thumb|left|300px|Example of two portals running in parallel, specific files are located in "ggbnportal" and "worldfloraonline". All other folders are used by both portal instances]]

*in common/config/bootstrap: the name of all folders that Yii2 should go through (for imports and so)
*in environnments/index.php: the name of the folders that will be exposed and therefore where Yii should be authorized to write logs and handle the CSS files

=Debug=
If something does not work as expected, have a look in the catalina.out file (generated by Tomcat) on the machine where SOLR is running. If Tomcat is running through XAMPP, you might have a part of the logs in the catalina.bat window. You will see the last queries sent to SOLR at the end of the file.
The YII file runtime/log/app.log might also help, and is located in the yii folder on the machine running the portal (ie. advanced/runtime....).

GGBN specific configuration

2016-02-09T16:06:46Z

GabiDroege: /* GGBN specific configuration */

GGBN specific configuration

2016-02-09T16:06:34Z

GabiDroege: Created page with "=GGBN specific configuration= * Several views are required for the counts-box on start page. <pre> CREATE ALGORITHM = UNDEFINED DEFINER = `root`@`localhost` SQL..."

=GGBN specific configuration=
* Several views are required for the counts-box on start page.
<pre>
CREATE
ALGORITHM = UNDEFINED
DEFINER = `root`@`localhost`
SQL SECURITY DEFINER
VIEW `counts` AS
select
count(`ro`.`occurrenceid`) AS `counts`, 'DNA' AS `kind`
from
(`ggbn_index`.`rawoccurrence` `ro`
join `ggbn_index`.`unitkind` `uk` ON ((`ro`.`fk_kindofunitid` = `uk`.`unitkindid`)))
where
(`uk`.`kindofunit_clean` = 'DNA')
union select
count(`ro`.`occurrenceid`) AS `counts`, 'Tissues' AS `kind`
from
(`ggbn_index`.`rawoccurrence` `ro`
join `ggbn_index`.`unitkind` `uk` ON ((`ro`.`fk_kindofunitid` = `uk`.`unitkindid`)))
where
(`uk`.`kindofunit_clean` = 'tissue')
union select
count(`ro`.`occurrenceid`) AS `counts`, 'Cultures' AS `kind`
from
(`ggbn_index`.`rawoccurrence` `ro`
join `ggbn_index`.`unitkind` `uk` ON ((`ro`.`fk_kindofunitid` = `uk`.`unitkindid`)))
where
(`uk`.`kindofunit_clean` = 'Culture')
union select
count(`ro`.`occurrenceid`) AS `counts`,
'eVouchers' AS `kind`
from
(`ggbn_index`.`rawoccurrence` `ro`
join `ggbn_index`.`unitkind` `uk` ON ((`ro`.`fk_kindofunitid` = `uk`.`unitkindid`)))
where
(`uk`.`kindofunit_clean` = 'eVoucher')
union select
count(`ro`.`occurrenceid`) AS `counts`,
'Specimens' AS `kind`
from
(`ggbn_index`.`rawoccurrence` `ro`
join `ggbn_index`.`unitkind` `uk` ON ((`ro`.`fk_kindofunitid` = `uk`.`unitkindid`)))
where
(`uk`.`kindofunit_clean` = 'specimen')
union select
count(`ro`.`occurrenceid`) AS `counts`, 'Unknown' AS `kind`
from
(`ggbn_index`.`rawoccurrence` `ro`
join `ggbn_index`.`unitkind` `uk` ON ((`ro`.`fk_kindofunitid` = `uk`.`unitkindid`)))
where
(`uk`.`kindofunit_clean` = 'unknown')
union select
count(`ro`.`occurrenceid`) AS `counts`, 'Enviros' AS `kind`
from
(`ggbn_index`.`rawoccurrence` `ro`
join `ggbn_index`.`unitkind` `uk` ON ((`ro`.`fk_kindofunitid` = `uk`.`unitkindid`)))
where
(`uk`.`kindofunit_clean` = 'environmental sample')
union select
count(`grouped_fullscientificname`.`fullscientificname`) AS `counts`,
'Taxa' AS `kind`
from
`ggbn_index`.`grouped_fullscientificname`
union select
count(distinct `families_view`.`family`) AS `counts`,
'Families' AS `kind`
from
`ggbn_index`.`families_view`
union select
count(distinct `genera_view`.`genus`) AS `counts`,
'Genera' AS `kind`
from
`ggbn_index`.`genera_view`
union select
count(distinct `species_view`.`species`) AS `counts`,
'Species' AS `kind`
from
`ggbn_index`.`species_view`
</pre>
<pre>
CREATE
ALGORITHM = UNDEFINED
DEFINER = `root`@`localhost`
SQL SECURITY DEFINER
VIEW `families_view` AS
select distinct
`f`.`family` AS `family`
from
((`backbone`.`family` `f`
join `backbone`.`name` `n` ON ((`f`.`familykey` = `n`.`familyKey`)))
join `ggbn_index`.`identification` `i` ON ((`i`.`gbifKey` = `n`.`acceptedKey`)))
union select distinct
`f`.`family` AS `family`
from
((`backbone_col`.`family` `f`
join `backbone_col`.`name` `n` ON ((`f`.`familykey` = `n`.`familyKey`)))
join `ggbn_index`.`identification` `i` ON ((`i`.`colKey` = `n`.`acceptedKey`)))
union select distinct
`f`.`family` AS `family`
from
((`backbone_ncbi`.`family` `f`
join `backbone_ncbi`.`name` `n` ON ((`f`.`familykey` = `n`.`familyKey`)))
join `ggbn_index`.`identification` `i` ON ((`i`.`ncbiKey` = `n`.`acceptedKey`)))
</pre>
<pre>
CREATE
ALGORITHM = UNDEFINED
DEFINER = `root`@`localhost`
SQL SECURITY DEFINER
VIEW `genera_view` AS
select distinct
`f`.`genus` AS `genus`
from
((`backbone`.`genus` `f`
join `backbone`.`name` `n` ON ((`f`.`genuskey` = `n`.`genusKey`)))
join `ggbn_index`.`identification` `i` ON ((`i`.`gbifKey` = `n`.`acceptedKey`)))
union select distinct
`f`.`genus` AS `genus`
from
((`backbone_col`.`genus` `f`
join `backbone_col`.`name` `n` ON ((`f`.`genuskey` = `n`.`genusKey`)))
join `ggbn_index`.`identification` `i` ON ((`i`.`colKey` = `n`.`acceptedKey`)))
union select distinct
`f`.`genus` AS `genus`
from
((`backbone_ncbi`.`genus` `f`
join `backbone_ncbi`.`name` `n` ON ((`f`.`genuskey` = `n`.`genusKey`)))
join `ggbn_index`.`identification` `i` ON ((`i`.`ncbiKey` = `n`.`acceptedKey`)))
</pre>
<pre>
CREATE
ALGORITHM = UNDEFINED
DEFINER = `root`@`localhost`
SQL SECURITY DEFINER
VIEW `grouped_fullscientificname` AS
select
`identification`.`fullScientificName` AS `fullscientificname`
from
`identification`
group by `identification`.`fullScientificName`
</pre>
<pre>
CREATE
ALGORITHM = UNDEFINED
DEFINER = `root`@`localhost`
SQL SECURITY DEFINER
VIEW `species_view` AS
select distinct
`n2`.`canonicalName` AS `species`
from
((`backbone`.`name` `n`
join `backbone`.`name` `n2` ON ((`n`.`acceptedKey` = `n2`.`namekey`)))
join `ggbn_index`.`identification` `i` ON ((`i`.`gbifKey` = `n`.`acceptedKey`)))
where
((`n`.`rank` = 'SPECIES')
and (`n2`.`rank` = 'SPECIES'))
union select distinct
`n2`.`canonicalName` AS `species`
from
((`backbone_col`.`name` `n`
join `backbone_col`.`name` `n2` ON ((`n`.`acceptedKey` = `n2`.`namekey`)))
join `ggbn_index`.`identification` `i` ON ((`i`.`colKey` = `n`.`acceptedKey`)))
where
((`n`.`rank` = 'SPECIES')
and (`n2`.`rank` = 'SPECIES'))
union select distinct
`n2`.`canonicalName` AS `species`
from
((`backbone_ncbi`.`name` `n`
join `backbone_ncbi`.`name` `n2` ON ((`n`.`acceptedKey` = `n2`.`namekey`)))
join `ggbn_index`.`identification` `i` ON ((`i`.`ncbiKey` = `n`.`acceptedKey`)))
where
((`n`.`rank` = 'SPECIES')
and (`n2`.`rank` = 'SPECIES'))
</pre>
* The institution has to be filled in bio_datasource (city and institution, for the harvester_factoy like '%Harvester%') - but first the datasource has to be added (metadata update). It will be used for the repository/registry field in the GGBN portal.
* The parentInstitution table is used for the contacts and must be filled manually.
* The GGBN registry is based on the NCD software and requires an additional database connection in the config file. Contact us for more information.
* For user management (login/shopping), a few extra tables are required. Follow the instructions from the file common/controllers/CommonPermissionController.php to create and populate them if you do not already have them from the SQL-Dump. You will also need an extra view for the statistics:
<pre>
CREATE
ALGORITHM = UNDEFINED
DEFINER = `root`@`localhost`
SQL SECURITY DEFINER
VIEW `shopping_view` AS
select distinct
`oc`.`fk_tripleidstoreid` AS `tripleidstoreid`,
`bs`.`id` AS `data_source_id`,
`pit`.`parentInstitutionID` AS `parentInstitutionID`,
`pit`.`institutionShort` AS `institutionShort`,
`pit`.`logoURL` AS `logoURL`
from
((`bio_datasource` `bs`
join `occurrence` `oc` ON ((`bs`.`id` = `oc`.`fk_datasourceid`)))
join `parentInstitution` `pit` ON ((`bs`.`fk_parentInstitutionid` = `pit`.`parentInstitutionID`)))
</pre>

Main Page

2016-02-05T13:52:41Z

GabiDroege: /* Introduction */

'''Welcome to the Wiki of the GGBN Portal Software'''

Further information about the software will follow soon.

=Introduction=
The GGBN Portal Software (GPS) has been developed by the Botanic Garden and Botanical Museum Berlin-Dahlem for the Global Genome Biodiversity Network ([http://www.ggbn.org GGBN]). The project was funded by the German Research Foundation (DFG (01/2014 - 03/2016)). The software is written in php using the [http://www.yiiframework.com/ Yii framework] and fully open source. The GPS allows costumization and therefore can be used for other special interest networks too. It is optimized for an underlying SOLR instance, data are harvested with [http://wiki.bgbm.org/bhit/ B-HIT].

=Workflow=
[[File:Workflow GPS.jpg|500px]]

Data are provided through BioCASe and IPT. The harvester [http://wiki.bgbm.org/bhit B-HIT] stores all original data in a MySQL database. To find out more about B-HIT please check out the [http://wiki.bgbm.org/bhit B-HIT Wiki]. After harvesting data are cleaned and can be matched against different taxonomic backbones. A SOLR instance is created on top of the MySQL database. The GPS itself requires access to the SOLR index as well as the MySQL index.

=Installation=

==You will need==
* Apache (tested with Apache/2.4.7 (Ubuntu), Apache/2.2.22 (Debian), Apache/2.4.10 (Win64)) with libapache2-mod-php5
* PHP5 (from 5.5! tested with 5.5.9, 5.5.30 and 5.5.15). You also might need to install php5-xsl, php5-gd and php5-mysql.
* a SOLR instance (tested with Apache SOLR 4.9.0). Contact us for getting the appropriate SOLR configuration files.
* a MySQL database (tested with MySQL 5.5.46). It will contain data harvested with [http://wiki.bgbm.org/bhit/index.php/Main_Page B-HIT]

Yii (version 2) will be fully contained in the subversion code of the portal software.

==Copy a working version from the svn ==
Create a destination folder, for the rest of this documentation we will choose "my_portal" in your web-folder (ie. /var/www/, or *WINPATH*). 
Get http://ww2.biocase.org/svn/dnabank/Dnabank_Portal/ggbn_portal/trunk/ into "my_portal".

=Configuration=
''TBD''
''yii, Apache, rights, SOLR (SOLR config files also & Java)'' 
First, let say we want to access the portal under the URL http://localhost/ggbn_portal.

*Check that the Apache User has writing/editing rights in the whole "my_portal" folder.
*Edit the file "my_portal/frontend/config/main.php".
**Configure the portal URL: <code>$baseUrl = str_replace ( '/SUBDIR_AFTER_YOUR_WEB_FOLDER/my_portal/frontend/web', '/ggbn_portal', (new Request ())->getBaseUrl () );</code>
**Configure the MySQL database credentials <code>
'db' => [
'class' => '\yii\db\Connection',
'dsn' => 'mysql:host=IP;dbname=DBNAME',
'username' => 'USER',
'password' => 'PASS',
'charset' => 'utf8'
],</code>
*Edit the file "my_portal/frontend/config/params.php".
**e-mails
***Configure the admin e-mail <code>'adminEmail' => 'ADMIN@DOMAIN',</code>. The admin will get error/bug messages.
***Configure the support e-mail(s) <code>'supportEmail' => ['MAIL1','MAIL2'..],</code>.
***Configure the feedback e-mail(s) <code>'feedbackMail' => ['MAIL1','MAIL2'..],</code>. They will get the messages sent with the feedback function.
***Configure the no-reply-mail (who is supposed to sent e-mail(s) to the users) <code>'noreplyMail'=>'MAIL'
**Configure the location of the DarwinCore Archives (harvested with B-HIT) <code>'dwcArchivesPath' => 'PATH_TO_THE_PARENT_FOLDER' ,</code>. Check the access rights! (read is enough)
**Configure the location(root) of your SOLR instance <code>'solrhost'=> 'SOLR_HOST'</code> and the SOLR core names
<code>
$searchname="SEARCH_CORE";
$previewname="PREVIEW_CORE";
$detailsname="DETAILS_CORE";
</code>
**Set the temporary folder <code>'tmpFolder'=>'TEMP_FOLDER'</code>. Check the access rights! (write is needed)
**Configure the options
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'noBack'=>true_OR_false</code>: a special backbone is used
***<code>'shopping'=>true_OR_false</code>: the shopping system is enabled or disabled
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'newsService'=>true_OR_false</code>: uses the GGBN news services - if it has to be used with a different wiki news, edit the SiteController.py
***<code>'displayMore'=>true_OR_false</code>: displays the categories with internal links on the details page
***<code>'displayExternals'=>true_OR_false</code>: displays the categories with external links (GBIF, BOLD, NCBI...) on the details page
***<code>'ABCDdownloable'=>true_OR_false</code>: ABCD files are downloadable or not
***<code>'useAnnosys'=>true_OR_false</code>: link to annosys (annotation system) on the details page
***<code>'annosysAP'=>'https://annosys.bgbm.fu-berlin.de/AnnoSys/AnnoSys?providerURL='</code>: root URL for Annosys
***<code>'siteName'=>'my_portal',</code>: the portal name (used in the URL)
***<code>'siteTitle'=>'GGBN Portal',</code>: the default site name/title
***<code>'catalogNumberClickable'=>true,</code>: if the catalogNumber/UnitID can be clickable on the details page

*Edit the Apache configuration file, for example:
<pre>
Alias /ggbn_portal "/var/www/my_portal/frontend/web"
<Directory "/var/www/my_portal/frontend/web">
Options All
AllowOverride All
</Directory>
</pre>

=Personalisation=
If you want to change the icon (favicon), it is located in frontend/web. 
If you want to change the slider images on the home page, they are located in frontend/web/images/slider. Then you will have to modify the view in frontend/view/site/index.php to load the pictures you chose. 
If you want to change the menu icons, they are located in frontend/web/images/icons. 
 
The CSS and JS files are also in the frontend/web folder.

=Adding new search fields or fields to display=
==Adding them in SOLR==
*edit the schema.xml file in SOLR and add the new fields you want to have. If you want to search for them, edit the file for the search core *_search/conf/schema.xml. If wou want to display them in the preview, edit the file for the preview core *_preview/conf/schema.xml. If you need it for the details page, edit the file for the details core *_details/conf/schema.xml. Generally: if you want to search for it (search core), the field has to be indexed, but not mandatory stored. If you want to display it (preview/details), it has to be stored but not indexed. The tripleidstoreid or the occurrenceid *always* has to be indexed.
*restart Tomcat
'''Important ''': always restart Tomcat (service tomcat8 restart) when you modify the schema.xml file.
*edit the Java file corresponding to the core that need to be updated- for example, if you want to index or store the recordbasis, add the following method
<code>
private void addRecordbasis(int datasourceid) throws ClassNotFoundException, SQLException {
System.out.println("RUN RECORDBASIS ");
if(tripleidstoreids.size()>0){
String queryTID =
"SELECT distinct fk_tripleidstoreid as tripleidstoreid, "
+ "IFNULL(recordbasis,'N/A') as recordbasis "
+ " FROM recordbasis "
+ " JOIN occurrence ON recordbasisid = fk_recordbasisid "
+ " WHERE fk_tripleidstoreid in ("+StringUtils.join(tripleidstoreids,", ")+") "
+ " AND fk_datasourceid ="+datasourceid
+ " ORDER BY fk_tripleidstoreid ";
update( queryTID,tripleidstoreids.size());
}
System.gc();
}
</code> and add a call to that method in the '''handleDocuments''' method:
<code>
occurrencesDone.clear();
multivalueForbidden=false;
addRecordbasis(datasourceid );
</code>
*existing documents can not be updated in SOLR in our case (only if everything was "stored"). So first, delete the old data from the core. Depending on how many records there are, you might have to delete the data for a specific repository only solrServer:solrPort/solr/coreName_SEARCH_OR_PREVIEW_OR_DETAILS/update?stream.body=<delete><query>YOUR_LIMITATION_FIELD:*</query></delete>&commit=true This "limitation_field" must be an indexed value in the core. If you don't have a lot of data, you can put <query>*:*</query> as the generation of the new SOLR index will be fast enough (5-10min per Core for 180.000 records).
*per default, it will index this new field for each datasource. If it's still missing at the end for some datasources, or if you only want to do it for specific datasources, edit the '''main''' method <code>for (int i=0;i<166;i++)</code>

==Adding them in the portal==
*Edit the SearchController file (controllers)(additions can be made in the CommonSearchController, deletion only in the specific SearchController if you have several portals running in parallel with the same installation)
**if it's a dropdownlist, add the name of the field in the variable <code>$knownLParams</code>.
**if it's a simple term or a suggestion list, add the name of the field in the variable <code>$knownParams</code>.
*Edit the SearchForm file (models)(same rule as above), edit the variable <code>$_parameters</code> - it can be a text, a select(dropdownlist), a suggest, a radio or a range. Also add the public variable with the same name (ie. <code>public $sampleavailability;</code> and
<code>
'sampleavailability' => [
'type' => 'radio',
'name' => 'Sample availability (loan)',
'config' => [
'size' => 285,
'maxlength' => 150,
'name' => 'sampleavailability'
],
'hideable' => 'true'
]
</code>. It will be automatically displayed in the search form. The '''hideable''' parameter specifies if it is displayed per default or if it is only in the accordion.
Eventually, edit the '''attribteLabels''' function such as <code>'sampleavailability' => 'Sample availability (loan)'</code>
*Edit the listView.php (views) file if it has to be displayed here, or the recordViewTabs.php if it has to be displayed on the details page. Here again, add it to the $suggestFields or $defaultFields or $selectFields or $buttonFields, depending of its type.
*Edit the file CreateFormManager(vendor) only if it's a radio button (look how it's done for cites and sampleavailability).
*Edit the file SOLRQueryManager(vendor) - for example, if it has to look in the SOLR case-unsensitive value, do as for
<code>
case 'fullScientificName' :
return $field . '_nc:' . $value . '';
break;
</code> If it's case sensitive,
<code>
case 'collectioncode' :
return $field . ':"' . $value . '"';
break;
</code>

=Running multiple GPS portals in parallel=
[[File:gps_folders.jpg|thumb|left|300px|Example of two portals running in parallel, specific files are located in "ggbnportal" and "worldfloraonline". All other folders are used by both portal instances]]

*in common/config/bootstrap: the name of all folders that Yii2 should go through (for imports and so)
*in environnments/index.php: the name of the folders that will be exposed and therefore where Yii should be authorized to write logs and handle the CSS files

=GGBN specific configuration=
* Several views are required for the counts-box on start page.
* The institution has to be filled in bio_datasource (city and institution, for the harvester_factoy like '%Harvester%') - but first the datasource has to be added (metadata update). It will be used for the repository/registry field in the GGBN portal.
* The parentInstitution table is used for the contacts and must be filled manually.
* The GGBN registry is based on the NCD software and requires an additional database connection in the config file.

=Debug=
If something does not work as expected, have a look in the catalina.out file (generated by Tomcat) on the machine where SOLR is running. You will see the last queries sent to SOLR at the end of the file.
The YII file runtime/log/app.log might also help, and is located in the yii folder on the machine running the portal (ie. advanced/runtime....).

Main Page

2016-02-05T13:51:21Z

GabiDroege: /* Running multiple GPS portals in parallel */

'''Welcome to the Wiki of the GGBN Portal Software'''

Further information about the software will follow soon.

=Introduction=
The GGBN Portal Software (GPS) has been developed by the Botanic Garden and Botanical Museum Berlin-Dahlem for the Global Genome Biodiversity Network ([http://www.ggbn.org GGBN]). The project was funded by the German Research Foundation (DFG (01/2014 - 03/2016)). The software is written in php using the Yii framework and fully open source. The GPS allows costumization and therefore can be used for other special interest networks too. It is optimized for an underlying SOLR instance, data are harvested with [http://wiki.bgbm.org/bhit/ B-HIT].

=Workflow=
[[File:Workflow GPS.jpg|500px]]

Data are provided through BioCASe and IPT. The harvester [http://wiki.bgbm.org/bhit B-HIT] stores all original data in a MySQL database. To find out more about B-HIT please check out the [http://wiki.bgbm.org/bhit B-HIT Wiki]. After harvesting data are cleaned and can be matched against different taxonomic backbones. A SOLR instance is created on top of the MySQL database. The GPS itself requires access to the SOLR index as well as the MySQL index.

=Installation=

==You will need==
* Apache (tested with Apache/2.4.7 (Ubuntu), Apache/2.2.22 (Debian), Apache/2.4.10 (Win64)) with libapache2-mod-php5
* PHP5 (from 5.5! tested with 5.5.9, 5.5.30 and 5.5.15). You also might need to install php5-xsl, php5-gd and php5-mysql.
* a SOLR instance (tested with Apache SOLR 4.9.0). Contact us for getting the appropriate SOLR configuration files.
* a MySQL database (tested with MySQL 5.5.46). It will contain data harvested with [http://wiki.bgbm.org/bhit/index.php/Main_Page B-HIT]

Yii (version 2) will be fully contained in the subversion code of the portal software.

==Copy a working version from the svn ==
Create a destination folder, for the rest of this documentation we will choose "my_portal" in your web-folder (ie. /var/www/, or *WINPATH*). 
Get http://ww2.biocase.org/svn/dnabank/Dnabank_Portal/ggbn_portal/trunk/ into "my_portal".

=Configuration=
''TBD''
''yii, Apache, rights, SOLR (SOLR config files also & Java)'' 
First, let say we want to access the portal under the URL http://localhost/ggbn_portal.

*Check that the Apache User has writing/editing rights in the whole "my_portal" folder.
*Edit the file "my_portal/frontend/config/main.php".
**Configure the portal URL: <code>$baseUrl = str_replace ( '/SUBDIR_AFTER_YOUR_WEB_FOLDER/my_portal/frontend/web', '/ggbn_portal', (new Request ())->getBaseUrl () );</code>
**Configure the MySQL database credentials <code>
'db' => [
'class' => '\yii\db\Connection',
'dsn' => 'mysql:host=IP;dbname=DBNAME',
'username' => 'USER',
'password' => 'PASS',
'charset' => 'utf8'
],</code>
*Edit the file "my_portal/frontend/config/params.php".
**e-mails
***Configure the admin e-mail <code>'adminEmail' => 'ADMIN@DOMAIN',</code>. The admin will get error/bug messages.
***Configure the support e-mail(s) <code>'supportEmail' => ['MAIL1','MAIL2'..],</code>.
***Configure the feedback e-mail(s) <code>'feedbackMail' => ['MAIL1','MAIL2'..],</code>. They will get the messages sent with the feedback function.
***Configure the no-reply-mail (who is supposed to sent e-mail(s) to the users) <code>'noreplyMail'=>'MAIL'
**Configure the location of the DarwinCore Archives (harvested with B-HIT) <code>'dwcArchivesPath' => 'PATH_TO_THE_PARENT_FOLDER' ,</code>. Check the access rights! (read is enough)
**Configure the location(root) of your SOLR instance <code>'solrhost'=> 'SOLR_HOST'</code> and the SOLR core names
<code>
$searchname="SEARCH_CORE";
$previewname="PREVIEW_CORE";
$detailsname="DETAILS_CORE";
</code>
**Set the temporary folder <code>'tmpFolder'=>'TEMP_FOLDER'</code>. Check the access rights! (write is needed)
**Configure the options
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'noBack'=>true_OR_false</code>: a special backbone is used
***<code>'shopping'=>true_OR_false</code>: the shopping system is enabled or disabled
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'newsService'=>true_OR_false</code>: uses the GGBN news services - if it has to be used with a different wiki news, edit the SiteController.py
***<code>'displayMore'=>true_OR_false</code>: displays the categories with internal links on the details page
***<code>'displayExternals'=>true_OR_false</code>: displays the categories with external links (GBIF, BOLD, NCBI...) on the details page
***<code>'ABCDdownloable'=>true_OR_false</code>: ABCD files are downloadable or not
***<code>'useAnnosys'=>true_OR_false</code>: link to annosys (annotation system) on the details page
***<code>'annosysAP'=>'https://annosys.bgbm.fu-berlin.de/AnnoSys/AnnoSys?providerURL='</code>: root URL for Annosys
***<code>'siteName'=>'my_portal',</code>: the portal name (used in the URL)
***<code>'siteTitle'=>'GGBN Portal',</code>: the default site name/title
***<code>'catalogNumberClickable'=>true,</code>: if the catalogNumber/UnitID can be clickable on the details page

*Edit the Apache configuration file, for example:
<pre>
Alias /ggbn_portal "/var/www/my_portal/frontend/web"
<Directory "/var/www/my_portal/frontend/web">
Options All
AllowOverride All
</Directory>
</pre>

=Personalisation=
If you want to change the icon (favicon), it is located in frontend/web. 
If you want to change the slider images on the home page, they are located in frontend/web/images/slider. Then you will have to modify the view in frontend/view/site/index.php to load the pictures you chose. 
If you want to change the menu icons, they are located in frontend/web/images/icons. 
 
The CSS and JS files are also in the frontend/web folder.

=Adding new search fields or fields to display=
==Adding them in SOLR==
*edit the schema.xml file in SOLR and add the new fields you want to have. If you want to search for them, edit the file for the search core *_search/conf/schema.xml. If wou want to display them in the preview, edit the file for the preview core *_preview/conf/schema.xml. If you need it for the details page, edit the file for the details core *_details/conf/schema.xml. Generally: if you want to search for it (search core), the field has to be indexed, but not mandatory stored. If you want to display it (preview/details), it has to be stored but not indexed. The tripleidstoreid or the occurrenceid *always* has to be indexed.
*restart Tomcat
'''Important ''': always restart Tomcat (service tomcat8 restart) when you modify the schema.xml file.
*edit the Java file corresponding to the core that need to be updated- for example, if you want to index or store the recordbasis, add the following method
<code>
private void addRecordbasis(int datasourceid) throws ClassNotFoundException, SQLException {
System.out.println("RUN RECORDBASIS ");
if(tripleidstoreids.size()>0){
String queryTID =
"SELECT distinct fk_tripleidstoreid as tripleidstoreid, "
+ "IFNULL(recordbasis,'N/A') as recordbasis "
+ " FROM recordbasis "
+ " JOIN occurrence ON recordbasisid = fk_recordbasisid "
+ " WHERE fk_tripleidstoreid in ("+StringUtils.join(tripleidstoreids,", ")+") "
+ " AND fk_datasourceid ="+datasourceid
+ " ORDER BY fk_tripleidstoreid ";
update( queryTID,tripleidstoreids.size());
}
System.gc();
}
</code> and add a call to that method in the '''handleDocuments''' method:
<code>
occurrencesDone.clear();
multivalueForbidden=false;
addRecordbasis(datasourceid );
</code>
*existing documents can not be updated in SOLR in our case (only if everything was "stored"). So first, delete the old data from the core. Depending on how many records there are, you might have to delete the data for a specific repository only solrServer:solrPort/solr/coreName_SEARCH_OR_PREVIEW_OR_DETAILS/update?stream.body=<delete><query>YOUR_LIMITATION_FIELD:*</query></delete>&commit=true This "limitation_field" must be an indexed value in the core. If you don't have a lot of data, you can put <query>*:*</query> as the generation of the new SOLR index will be fast enough (5-10min per Core for 180.000 records).
*per default, it will index this new field for each datasource. If it's still missing at the end for some datasources, or if you only want to do it for specific datasources, edit the '''main''' method <code>for (int i=0;i<166;i++)</code>

==Adding them in the portal==
*Edit the SearchController file (controllers)(additions can be made in the CommonSearchController, deletion only in the specific SearchController if you have several portals running in parallel with the same installation)
**if it's a dropdownlist, add the name of the field in the variable <code>$knownLParams</code>.
**if it's a simple term or a suggestion list, add the name of the field in the variable <code>$knownParams</code>.
*Edit the SearchForm file (models)(same rule as above), edit the variable <code>$_parameters</code> - it can be a text, a select(dropdownlist), a suggest, a radio or a range. Also add the public variable with the same name (ie. <code>public $sampleavailability;</code> and
<code>
'sampleavailability' => [
'type' => 'radio',
'name' => 'Sample availability (loan)',
'config' => [
'size' => 285,
'maxlength' => 150,
'name' => 'sampleavailability'
],
'hideable' => 'true'
]
</code>. It will be automatically displayed in the search form. The '''hideable''' parameter specifies if it is displayed per default or if it is only in the accordion.
Eventually, edit the '''attribteLabels''' function such as <code>'sampleavailability' => 'Sample availability (loan)'</code>
*Edit the listView.php (views) file if it has to be displayed here, or the recordViewTabs.php if it has to be displayed on the details page. Here again, add it to the $suggestFields or $defaultFields or $selectFields or $buttonFields, depending of its type.
*Edit the file CreateFormManager(vendor) only if it's a radio button (look how it's done for cites and sampleavailability).
*Edit the file SOLRQueryManager(vendor) - for example, if it has to look in the SOLR case-unsensitive value, do as for
<code>
case 'fullScientificName' :
return $field . '_nc:' . $value . '';
break;
</code> If it's case sensitive,
<code>
case 'collectioncode' :
return $field . ':"' . $value . '"';
break;
</code>

=Running multiple GPS portals in parallel=
[[File:gps_folders.jpg|thumb|left|300px|Example of two portals running in parallel, specific files are located in "ggbnportal" and "worldfloraonline". All other folders are used by both portal instances]]

*in common/config/bootstrap: the name of all folders that Yii2 should go through (for imports and so)
*in environnments/index.php: the name of the folders that will be exposed and therefore where Yii should be authorized to write logs and handle the CSS files

=GGBN specific configuration=
* Several views are required for the counts-box on start page.
* The institution has to be filled in bio_datasource (city and institution, for the harvester_factoy like '%Harvester%') - but first the datasource has to be added (metadata update). It will be used for the repository/registry field in the GGBN portal.
* The parentInstitution table is used for the contacts and must be filled manually.
* The GGBN registry is based on the NCD software and requires an additional database connection in the config file.

=Debug=
If something does not work as expected, have a look in the catalina.out file (generated by Tomcat) on the machine where SOLR is running. You will see the last queries sent to SOLR at the end of the file.
The YII file runtime/log/app.log might also help, and is located in the yii folder on the machine running the portal (ie. advanced/runtime....).

Main Page

2016-02-05T13:51:00Z

GabiDroege: /* Running multiple GPS portals in parallel */

'''Welcome to the Wiki of the GGBN Portal Software'''

Further information about the software will follow soon.

=Introduction=
The GGBN Portal Software (GPS) has been developed by the Botanic Garden and Botanical Museum Berlin-Dahlem for the Global Genome Biodiversity Network ([http://www.ggbn.org GGBN]). The project was funded by the German Research Foundation (DFG (01/2014 - 03/2016)). The software is written in php using the Yii framework and fully open source. The GPS allows costumization and therefore can be used for other special interest networks too. It is optimized for an underlying SOLR instance, data are harvested with [http://wiki.bgbm.org/bhit/ B-HIT].

=Workflow=
[[File:Workflow GPS.jpg|500px]]

Data are provided through BioCASe and IPT. The harvester [http://wiki.bgbm.org/bhit B-HIT] stores all original data in a MySQL database. To find out more about B-HIT please check out the [http://wiki.bgbm.org/bhit B-HIT Wiki]. After harvesting data are cleaned and can be matched against different taxonomic backbones. A SOLR instance is created on top of the MySQL database. The GPS itself requires access to the SOLR index as well as the MySQL index.

=Installation=

==You will need==
* Apache (tested with Apache/2.4.7 (Ubuntu), Apache/2.2.22 (Debian), Apache/2.4.10 (Win64)) with libapache2-mod-php5
* PHP5 (from 5.5! tested with 5.5.9, 5.5.30 and 5.5.15). You also might need to install php5-xsl, php5-gd and php5-mysql.
* a SOLR instance (tested with Apache SOLR 4.9.0). Contact us for getting the appropriate SOLR configuration files.
* a MySQL database (tested with MySQL 5.5.46). It will contain data harvested with [http://wiki.bgbm.org/bhit/index.php/Main_Page B-HIT]

Yii (version 2) will be fully contained in the subversion code of the portal software.

==Copy a working version from the svn ==
Create a destination folder, for the rest of this documentation we will choose "my_portal" in your web-folder (ie. /var/www/, or *WINPATH*). 
Get http://ww2.biocase.org/svn/dnabank/Dnabank_Portal/ggbn_portal/trunk/ into "my_portal".

=Configuration=
''TBD''
''yii, Apache, rights, SOLR (SOLR config files also & Java)'' 
First, let say we want to access the portal under the URL http://localhost/ggbn_portal.

*Check that the Apache User has writing/editing rights in the whole "my_portal" folder.
*Edit the file "my_portal/frontend/config/main.php".
**Configure the portal URL: <code>$baseUrl = str_replace ( '/SUBDIR_AFTER_YOUR_WEB_FOLDER/my_portal/frontend/web', '/ggbn_portal', (new Request ())->getBaseUrl () );</code>
**Configure the MySQL database credentials <code>
'db' => [
'class' => '\yii\db\Connection',
'dsn' => 'mysql:host=IP;dbname=DBNAME',
'username' => 'USER',
'password' => 'PASS',
'charset' => 'utf8'
],</code>
*Edit the file "my_portal/frontend/config/params.php".
**e-mails
***Configure the admin e-mail <code>'adminEmail' => 'ADMIN@DOMAIN',</code>. The admin will get error/bug messages.
***Configure the support e-mail(s) <code>'supportEmail' => ['MAIL1','MAIL2'..],</code>.
***Configure the feedback e-mail(s) <code>'feedbackMail' => ['MAIL1','MAIL2'..],</code>. They will get the messages sent with the feedback function.
***Configure the no-reply-mail (who is supposed to sent e-mail(s) to the users) <code>'noreplyMail'=>'MAIL'
**Configure the location of the DarwinCore Archives (harvested with B-HIT) <code>'dwcArchivesPath' => 'PATH_TO_THE_PARENT_FOLDER' ,</code>. Check the access rights! (read is enough)
**Configure the location(root) of your SOLR instance <code>'solrhost'=> 'SOLR_HOST'</code> and the SOLR core names
<code>
$searchname="SEARCH_CORE";
$previewname="PREVIEW_CORE";
$detailsname="DETAILS_CORE";
</code>
**Set the temporary folder <code>'tmpFolder'=>'TEMP_FOLDER'</code>. Check the access rights! (write is needed)
**Configure the options
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'noBack'=>true_OR_false</code>: a special backbone is used
***<code>'shopping'=>true_OR_false</code>: the shopping system is enabled or disabled
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'newsService'=>true_OR_false</code>: uses the GGBN news services - if it has to be used with a different wiki news, edit the SiteController.py
***<code>'displayMore'=>true_OR_false</code>: displays the categories with internal links on the details page
***<code>'displayExternals'=>true_OR_false</code>: displays the categories with external links (GBIF, BOLD, NCBI...) on the details page
***<code>'ABCDdownloable'=>true_OR_false</code>: ABCD files are downloadable or not
***<code>'useAnnosys'=>true_OR_false</code>: link to annosys (annotation system) on the details page
***<code>'annosysAP'=>'https://annosys.bgbm.fu-berlin.de/AnnoSys/AnnoSys?providerURL='</code>: root URL for Annosys
***<code>'siteName'=>'my_portal',</code>: the portal name (used in the URL)
***<code>'siteTitle'=>'GGBN Portal',</code>: the default site name/title
***<code>'catalogNumberClickable'=>true,</code>: if the catalogNumber/UnitID can be clickable on the details page

*Edit the Apache configuration file, for example:
<pre>
Alias /ggbn_portal "/var/www/my_portal/frontend/web"
<Directory "/var/www/my_portal/frontend/web">
Options All
AllowOverride All
</Directory>
</pre>

=Personalisation=
If you want to change the icon (favicon), it is located in frontend/web. 
If you want to change the slider images on the home page, they are located in frontend/web/images/slider. Then you will have to modify the view in frontend/view/site/index.php to load the pictures you chose. 
If you want to change the menu icons, they are located in frontend/web/images/icons. 
 
The CSS and JS files are also in the frontend/web folder.

=Adding new search fields or fields to display=
==Adding them in SOLR==
*edit the schema.xml file in SOLR and add the new fields you want to have. If you want to search for them, edit the file for the search core *_search/conf/schema.xml. If wou want to display them in the preview, edit the file for the preview core *_preview/conf/schema.xml. If you need it for the details page, edit the file for the details core *_details/conf/schema.xml. Generally: if you want to search for it (search core), the field has to be indexed, but not mandatory stored. If you want to display it (preview/details), it has to be stored but not indexed. The tripleidstoreid or the occurrenceid *always* has to be indexed.
*restart Tomcat
'''Important ''': always restart Tomcat (service tomcat8 restart) when you modify the schema.xml file.
*edit the Java file corresponding to the core that need to be updated- for example, if you want to index or store the recordbasis, add the following method
<code>
private void addRecordbasis(int datasourceid) throws ClassNotFoundException, SQLException {
System.out.println("RUN RECORDBASIS ");
if(tripleidstoreids.size()>0){
String queryTID =
"SELECT distinct fk_tripleidstoreid as tripleidstoreid, "
+ "IFNULL(recordbasis,'N/A') as recordbasis "
+ " FROM recordbasis "
+ " JOIN occurrence ON recordbasisid = fk_recordbasisid "
+ " WHERE fk_tripleidstoreid in ("+StringUtils.join(tripleidstoreids,", ")+") "
+ " AND fk_datasourceid ="+datasourceid
+ " ORDER BY fk_tripleidstoreid ";
update( queryTID,tripleidstoreids.size());
}
System.gc();
}
</code> and add a call to that method in the '''handleDocuments''' method:
<code>
occurrencesDone.clear();
multivalueForbidden=false;
addRecordbasis(datasourceid );
</code>
*existing documents can not be updated in SOLR in our case (only if everything was "stored"). So first, delete the old data from the core. Depending on how many records there are, you might have to delete the data for a specific repository only solrServer:solrPort/solr/coreName_SEARCH_OR_PREVIEW_OR_DETAILS/update?stream.body=<delete><query>YOUR_LIMITATION_FIELD:*</query></delete>&commit=true This "limitation_field" must be an indexed value in the core. If you don't have a lot of data, you can put <query>*:*</query> as the generation of the new SOLR index will be fast enough (5-10min per Core for 180.000 records).
*per default, it will index this new field for each datasource. If it's still missing at the end for some datasources, or if you only want to do it for specific datasources, edit the '''main''' method <code>for (int i=0;i<166;i++)</code>

==Adding them in the portal==
*Edit the SearchController file (controllers)(additions can be made in the CommonSearchController, deletion only in the specific SearchController if you have several portals running in parallel with the same installation)
**if it's a dropdownlist, add the name of the field in the variable <code>$knownLParams</code>.
**if it's a simple term or a suggestion list, add the name of the field in the variable <code>$knownParams</code>.
*Edit the SearchForm file (models)(same rule as above), edit the variable <code>$_parameters</code> - it can be a text, a select(dropdownlist), a suggest, a radio or a range. Also add the public variable with the same name (ie. <code>public $sampleavailability;</code> and
<code>
'sampleavailability' => [
'type' => 'radio',
'name' => 'Sample availability (loan)',
'config' => [
'size' => 285,
'maxlength' => 150,
'name' => 'sampleavailability'
],
'hideable' => 'true'
]
</code>. It will be automatically displayed in the search form. The '''hideable''' parameter specifies if it is displayed per default or if it is only in the accordion.
Eventually, edit the '''attribteLabels''' function such as <code>'sampleavailability' => 'Sample availability (loan)'</code>
*Edit the listView.php (views) file if it has to be displayed here, or the recordViewTabs.php if it has to be displayed on the details page. Here again, add it to the $suggestFields or $defaultFields or $selectFields or $buttonFields, depending of its type.
*Edit the file CreateFormManager(vendor) only if it's a radio button (look how it's done for cites and sampleavailability).
*Edit the file SOLRQueryManager(vendor) - for example, if it has to look in the SOLR case-unsensitive value, do as for
<code>
case 'fullScientificName' :
return $field . '_nc:' . $value . '';
break;
</code> If it's case sensitive,
<code>
case 'collectioncode' :
return $field . ':"' . $value . '"';
break;
</code>

=Running multiple GPS portals in parallel=

*in common/config/bootstrap: the name of all folders that Yii2 should go through (for imports and so)
*in environnments/index.php: the name of the folders that will be exposed and therefore where Yii should be authorized to write logs and handle the CSS files

[[File:gps_folders.jpg|thumb|left|400px|Example of two portals running in parallel, specific files are located in "ggbnportal" and "worldfloraonline". All other folders are used by both portal instances]]

=GGBN specific configuration=
* Several views are required for the counts-box on start page.
* The institution has to be filled in bio_datasource (city and institution, for the harvester_factoy like '%Harvester%') - but first the datasource has to be added (metadata update). It will be used for the repository/registry field in the GGBN portal.
* The parentInstitution table is used for the contacts and must be filled manually.
* The GGBN registry is based on the NCD software and requires an additional database connection in the config file.

=Debug=
If something does not work as expected, have a look in the catalina.out file (generated by Tomcat) on the machine where SOLR is running. You will see the last queries sent to SOLR at the end of the file.
The YII file runtime/log/app.log might also help, and is located in the yii folder on the machine running the portal (ie. advanced/runtime....).

Main Page

2016-02-05T13:49:03Z

GabiDroege: /* GGBN specific configuration */

'''Welcome to the Wiki of the GGBN Portal Software'''

Further information about the software will follow soon.

=Introduction=
The GGBN Portal Software (GPS) has been developed by the Botanic Garden and Botanical Museum Berlin-Dahlem for the Global Genome Biodiversity Network ([http://www.ggbn.org GGBN]). The project was funded by the German Research Foundation (DFG (01/2014 - 03/2016)). The software is written in php using the Yii framework and fully open source. The GPS allows costumization and therefore can be used for other special interest networks too. It is optimized for an underlying SOLR instance, data are harvested with [http://wiki.bgbm.org/bhit/ B-HIT].

=Workflow=
[[File:Workflow GPS.jpg|500px]]

Data are provided through BioCASe and IPT. The harvester [http://wiki.bgbm.org/bhit B-HIT] stores all original data in a MySQL database. To find out more about B-HIT please check out the [http://wiki.bgbm.org/bhit B-HIT Wiki]. After harvesting data are cleaned and can be matched against different taxonomic backbones. A SOLR instance is created on top of the MySQL database. The GPS itself requires access to the SOLR index as well as the MySQL index.

=Installation=

==You will need==
* Apache (tested with Apache/2.4.7 (Ubuntu), Apache/2.2.22 (Debian), Apache/2.4.10 (Win64)) with libapache2-mod-php5
* PHP5 (from 5.5! tested with 5.5.9, 5.5.30 and 5.5.15). You also might need to install php5-xsl, php5-gd and php5-mysql.
* a SOLR instance (tested with Apache SOLR 4.9.0). Contact us for getting the appropriate SOLR configuration files.
* a MySQL database (tested with MySQL 5.5.46). It will contain data harvested with [http://wiki.bgbm.org/bhit/index.php/Main_Page B-HIT]

Yii (version 2) will be fully contained in the subversion code of the portal software.

==Copy a working version from the svn ==
Create a destination folder, for the rest of this documentation we will choose "my_portal" in your web-folder (ie. /var/www/, or *WINPATH*). 
Get http://ww2.biocase.org/svn/dnabank/Dnabank_Portal/ggbn_portal/trunk/ into "my_portal".

=Configuration=
''TBD''
''yii, Apache, rights, SOLR (SOLR config files also & Java)'' 
First, let say we want to access the portal under the URL http://localhost/ggbn_portal.

*Check that the Apache User has writing/editing rights in the whole "my_portal" folder.
*Edit the file "my_portal/frontend/config/main.php".
**Configure the portal URL: <code>$baseUrl = str_replace ( '/SUBDIR_AFTER_YOUR_WEB_FOLDER/my_portal/frontend/web', '/ggbn_portal', (new Request ())->getBaseUrl () );</code>
**Configure the MySQL database credentials <code>
'db' => [
'class' => '\yii\db\Connection',
'dsn' => 'mysql:host=IP;dbname=DBNAME',
'username' => 'USER',
'password' => 'PASS',
'charset' => 'utf8'
],</code>
*Edit the file "my_portal/frontend/config/params.php".
**e-mails
***Configure the admin e-mail <code>'adminEmail' => 'ADMIN@DOMAIN',</code>. The admin will get error/bug messages.
***Configure the support e-mail(s) <code>'supportEmail' => ['MAIL1','MAIL2'..],</code>.
***Configure the feedback e-mail(s) <code>'feedbackMail' => ['MAIL1','MAIL2'..],</code>. They will get the messages sent with the feedback function.
***Configure the no-reply-mail (who is supposed to sent e-mail(s) to the users) <code>'noreplyMail'=>'MAIL'
**Configure the location of the DarwinCore Archives (harvested with B-HIT) <code>'dwcArchivesPath' => 'PATH_TO_THE_PARENT_FOLDER' ,</code>. Check the access rights! (read is enough)
**Configure the location(root) of your SOLR instance <code>'solrhost'=> 'SOLR_HOST'</code> and the SOLR core names
<code>
$searchname="SEARCH_CORE";
$previewname="PREVIEW_CORE";
$detailsname="DETAILS_CORE";
</code>
**Set the temporary folder <code>'tmpFolder'=>'TEMP_FOLDER'</code>. Check the access rights! (write is needed)
**Configure the options
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'noBack'=>true_OR_false</code>: a special backbone is used
***<code>'shopping'=>true_OR_false</code>: the shopping system is enabled or disabled
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'newsService'=>true_OR_false</code>: uses the GGBN news services - if it has to be used with a different wiki news, edit the SiteController.py
***<code>'displayMore'=>true_OR_false</code>: displays the categories with internal links on the details page
***<code>'displayExternals'=>true_OR_false</code>: displays the categories with external links (GBIF, BOLD, NCBI...) on the details page
***<code>'ABCDdownloable'=>true_OR_false</code>: ABCD files are downloadable or not
***<code>'useAnnosys'=>true_OR_false</code>: link to annosys (annotation system) on the details page
***<code>'annosysAP'=>'https://annosys.bgbm.fu-berlin.de/AnnoSys/AnnoSys?providerURL='</code>: root URL for Annosys
***<code>'siteName'=>'my_portal',</code>: the portal name (used in the URL)
***<code>'siteTitle'=>'GGBN Portal',</code>: the default site name/title
***<code>'catalogNumberClickable'=>true,</code>: if the catalogNumber/UnitID can be clickable on the details page

*Edit the Apache configuration file, for example:
<pre>
Alias /ggbn_portal "/var/www/my_portal/frontend/web"
<Directory "/var/www/my_portal/frontend/web">
Options All
AllowOverride All
</Directory>
</pre>

=Personalisation=
If you want to change the icon (favicon), it is located in frontend/web. 
If you want to change the slider images on the home page, they are located in frontend/web/images/slider. Then you will have to modify the view in frontend/view/site/index.php to load the pictures you chose. 
If you want to change the menu icons, they are located in frontend/web/images/icons. 
 
The CSS and JS files are also in the frontend/web folder.

=Adding new search fields or fields to display=
==Adding them in SOLR==
*edit the schema.xml file in SOLR and add the new fields you want to have. If you want to search for them, edit the file for the search core *_search/conf/schema.xml. If wou want to display them in the preview, edit the file for the preview core *_preview/conf/schema.xml. If you need it for the details page, edit the file for the details core *_details/conf/schema.xml. Generally: if you want to search for it (search core), the field has to be indexed, but not mandatory stored. If you want to display it (preview/details), it has to be stored but not indexed. The tripleidstoreid or the occurrenceid *always* has to be indexed.
*restart Tomcat
'''Important ''': always restart Tomcat (service tomcat8 restart) when you modify the schema.xml file.
*edit the Java file corresponding to the core that need to be updated- for example, if you want to index or store the recordbasis, add the following method
<code>
private void addRecordbasis(int datasourceid) throws ClassNotFoundException, SQLException {
System.out.println("RUN RECORDBASIS ");
if(tripleidstoreids.size()>0){
String queryTID =
"SELECT distinct fk_tripleidstoreid as tripleidstoreid, "
+ "IFNULL(recordbasis,'N/A') as recordbasis "
+ " FROM recordbasis "
+ " JOIN occurrence ON recordbasisid = fk_recordbasisid "
+ " WHERE fk_tripleidstoreid in ("+StringUtils.join(tripleidstoreids,", ")+") "
+ " AND fk_datasourceid ="+datasourceid
+ " ORDER BY fk_tripleidstoreid ";
update( queryTID,tripleidstoreids.size());
}
System.gc();
}
</code> and add a call to that method in the '''handleDocuments''' method:
<code>
occurrencesDone.clear();
multivalueForbidden=false;
addRecordbasis(datasourceid );
</code>
*existing documents can not be updated in SOLR in our case (only if everything was "stored"). So first, delete the old data from the core. Depending on how many records there are, you might have to delete the data for a specific repository only solrServer:solrPort/solr/coreName_SEARCH_OR_PREVIEW_OR_DETAILS/update?stream.body=<delete><query>YOUR_LIMITATION_FIELD:*</query></delete>&commit=true This "limitation_field" must be an indexed value in the core. If you don't have a lot of data, you can put <query>*:*</query> as the generation of the new SOLR index will be fast enough (5-10min per Core for 180.000 records).
*per default, it will index this new field for each datasource. If it's still missing at the end for some datasources, or if you only want to do it for specific datasources, edit the '''main''' method <code>for (int i=0;i<166;i++)</code>

==Adding them in the portal==
*Edit the SearchController file (controllers)(additions can be made in the CommonSearchController, deletion only in the specific SearchController if you have several portals running in parallel with the same installation)
**if it's a dropdownlist, add the name of the field in the variable <code>$knownLParams</code>.
**if it's a simple term or a suggestion list, add the name of the field in the variable <code>$knownParams</code>.
*Edit the SearchForm file (models)(same rule as above), edit the variable <code>$_parameters</code> - it can be a text, a select(dropdownlist), a suggest, a radio or a range. Also add the public variable with the same name (ie. <code>public $sampleavailability;</code> and
<code>
'sampleavailability' => [
'type' => 'radio',
'name' => 'Sample availability (loan)',
'config' => [
'size' => 285,
'maxlength' => 150,
'name' => 'sampleavailability'
],
'hideable' => 'true'
]
</code>. It will be automatically displayed in the search form. The '''hideable''' parameter specifies if it is displayed per default or if it is only in the accordion.
Eventually, edit the '''attribteLabels''' function such as <code>'sampleavailability' => 'Sample availability (loan)'</code>
*Edit the listView.php (views) file if it has to be displayed here, or the recordViewTabs.php if it has to be displayed on the details page. Here again, add it to the $suggestFields or $defaultFields or $selectFields or $buttonFields, depending of its type.
*Edit the file CreateFormManager(vendor) only if it's a radio button (look how it's done for cites and sampleavailability).
*Edit the file SOLRQueryManager(vendor) - for example, if it has to look in the SOLR case-unsensitive value, do as for
<code>
case 'fullScientificName' :
return $field . '_nc:' . $value . '';
break;
</code> If it's case sensitive,
<code>
case 'collectioncode' :
return $field . ':"' . $value . '"';
break;
</code>

=Running multiple GPS portals in parallel=

*in common/config/bootstrap: the name of all folders that Yii2 should go through (for imports and so)
*in environnments/index.php: the name of the folders that will be exposed and therefore where Yii should be authorized to write logs and handle the CSS files

[[File:gps_folders.jpg]]

=GGBN specific configuration=
* Several views are required for the counts-box on start page.
* The institution has to be filled in bio_datasource (city and institution, for the harvester_factoy like '%Harvester%') - but first the datasource has to be added (metadata update). It will be used for the repository/registry field in the GGBN portal.
* The parentInstitution table is used for the contacts and must be filled manually.
* The GGBN registry is based on the NCD software and requires an additional database connection in the config file.

=Debug=
If something does not work as expected, have a look in the catalina.out file (generated by Tomcat) on the machine where SOLR is running. You will see the last queries sent to SOLR at the end of the file.
The YII file runtime/log/app.log might also help, and is located in the yii folder on the machine running the portal (ie. advanced/runtime....).

Main Page

2016-02-03T14:20:47Z

GabiDroege: /* You will need */

'''Welcome to the Wiki of the GGBN Portal Software'''

Further information about the software will follow soon.

=Introduction=
The GGBN Portal Software (GPS) has been developed by the Botanic Garden and Botanical Museum Berlin-Dahlem for the Global Genome Biodiversity Network ([http://www.ggbn.org GGBN]). The project was funded by the German Research Foundation (DFG (01/2014 - 03/2016)). The software is written in php using the Yii framework and fully open source. The GPS allows costumization and therefore can be used for other special interest networks too. It is optimized for an underlying SOLR instance, data are harvested with [http://wiki.bgbm.org/bhit/ B-HIT].

=Workflow=
[[File:Workflow GPS.jpg|500px]]

Data are provided through BioCASe and IPT. The harvester [http://wiki.bgbm.org/bhit B-HIT] stores all original data in a MySQL database. To find out more about B-HIT please check out the [http://wiki.bgbm.org/bhit B-HIT Wiki]. After harvesting data are cleaned and can be matched against different taxonomic backbones. A SOLR instance is created on top of the MySQL database. The GPS itself requires access to the SOLR index as well as the MySQL index.

=Installation=

==You will need==
* Apache (tested with Apache/2.4.7 (Ubuntu), Apache/2.2.22 (Debian), Apache/2.4.10 (Win64)) with libapache2-mod-php5
* PHP5 (tested with 5.5.9, 5.5.30 and 5.5.15). You also might need to install php5-xsl, php5-gd and php5-mysql.
* a SOLR instance (tested with Apache SOLR 4.9.0). Contact us for getting the appropriate SOLR configuration files.
* a MySQL database (tested with MySQL 5.5.46). It will contain data harvested with [http://wiki.bgbm.org/bhit/index.php/Main_Page B-HIT]

Yii (version 2) will be fully contained in the subversion code of the portal software.

==Copy a working version from the svn ==
Create a destination folder, for the rest of this documentation we will choose "my_portal" in your web-folder (ie. /var/www/, or *WINPATH*). 
Get http://ww2.biocase.org/svn/dnabank/Dnabank_Portal/ggbn_portal/trunk/ into "my_portal".

=Configuration=
''TBD''
''yii, Apache, rights, SOLR (SOLR config files also & Java)'' 
First, let say we want to access the portal under the URL http://localhost/ggbn_portal.

*Check that the Apache User has writing/editing rights in the whole "my_portal" folder.
*Edit the file "my_portal/frontend/config/main.php".
**Configure the portal URL: <code>$baseUrl = str_replace ( '/SUBDIR_AFTER_YOUR_WEB_FOLDER/my_portal/frontend/web', '/ggbn_portal', (new Request ())->getBaseUrl () );</code>
**Configure the MySQL database credentials <code>
'db' => [
'class' => '\yii\db\Connection',
'dsn' => 'mysql:host=IP;dbname=DBNAME',
'username' => 'USER',
'password' => 'PASS',
'charset' => 'utf8'
],</code>
*Edit the file "my_portal/frontend/config/params.php".
**e-mails
***Configure the admin e-mail <code>'adminEmail' => 'ADMIN@DOMAIN',</code>. The admin will get error/bug messages.
***Configure the support e-mail(s) <code>'supportEmail' => ['MAIL1','MAIL2'..],</code>.
***Configure the feedback e-mail(s) <code>'feedbackMail' => ['MAIL1','MAIL2'..],</code>. They will get the messages sent with the feedback function.
***Configure the no-reply-mail (who is supposed to sent e-mail(s) to the users) <code>'noreplyMail'=>'MAIL'
**Configure the location of the DarwinCore Archives (harvested with B-HIT) <code>'dwcArchivesPath' => 'PATH_TO_THE_PARENT_FOLDER' ,</code>. Check the access rights! (read is enough)
**Configure the location(root) of your SOLR instance <code>'solrhost'=> 'SOLR_HOST'</code> and the SOLR core names
<code>
$searchname="SEARCH_CORE";
$previewname="PREVIEW_CORE";
$detailsname="DETAILS_CORE";
</code>
**Set the temporary folder <code>'tmpFolder'=>'TEMP_FOLDER'</code>. Check the access rights! (write is needed)
**Configure the options
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'noBack'=>true_OR_false</code>: a special backbone is used
***<code>'shopping'=>true_OR_false</code>: the shopping system is enabled or disabled
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'newsService'=>true_OR_false</code>: uses the GGBN news services - if it has to be used with a different wiki news, edit the SiteController.py
***<code>'displayMore'=>true_OR_false</code>: displays the categories with internal links on the details page
***<code>'displayExternals'=>true_OR_false</code>: displays the categories with external links (GBIF, BOLD, NCBI...) on the details page
***<code>'ABCDdownloable'=>true_OR_false</code>: ABCD files are downloadable or not
***<code>'useAnnosys'=>true_OR_false</code>: link to annosys (annotation system) on the details page
***<code>'annosysAP'=>'https://annosys.bgbm.fu-berlin.de/AnnoSys/AnnoSys?providerURL='</code>: root URL for Annosys
***<code>'siteName'=>'my_portal',</code>: the portal name (used in the URL)
***<code>'siteTitle'=>'GGBN Portal',</code>: the default site name/title
***<code>'catalogNumberClickable'=>true,</code>: if the catalogNumber/UnitID can be clickable on the details page

=Personalisation=
If you want to change the icon (favicon), it is located in frontend/web. 
If you want to change the slider images on the home page, they are located in frontend/web/images/slider. Then you will have to modify the view in frontend/view/site/index.php to load the pictures you chose. 
If you want to change the menu icons, they are located in frontend/web/images/icons. 
 
The CSS and JS files are also in the frontend/web folder.

=Adding new search fields or fields to display=
==Adding them in SOLR==
*edit the schema.xml file in SOLR and add the new fields you want to have. If you want to search for them, edit the file for the search core *_search/conf/schema.xml. If wou want to display them in the preview, edit the file for the preview core *_preview/conf/schema.xml. If you need it for the details page, edit the file for the details core *_details/conf/schema.xml. Generally: if you want to search for it (search core), the field has to be indexed, but not mandatory stored. If you want to display it (preview/details), it has to be stored but not indexed. The tripleidstoreid or the occurrenceid *always* has to be indexed.
*restart Tomcat
'''Important ''': always restart Tomcat (service tomcat8 restart) when you modify the schema.xml file.
*edit the Java file corresponding to the core that need to be updated- for example, if you want to index or store the recordbasis, add the following method
<code>
private void addRecordbasis(int datasourceid) throws ClassNotFoundException, SQLException {
System.out.println("RUN RECORDBASIS ");
if(tripleidstoreids.size()>0){
String queryTID =
"SELECT distinct fk_tripleidstoreid as tripleidstoreid, "
+ "IFNULL(recordbasis,'N/A') as recordbasis "
+ " FROM recordbasis "
+ " JOIN occurrence ON recordbasisid = fk_recordbasisid "
+ " WHERE fk_tripleidstoreid in ("+StringUtils.join(tripleidstoreids,", ")+") "
+ " AND fk_datasourceid ="+datasourceid
+ " ORDER BY fk_tripleidstoreid ";
update( queryTID,tripleidstoreids.size());
}
System.gc();
}
</code> and add a call to that method in the '''handleDocuments''' method:
<code>
occurrencesDone.clear();
multivalueForbidden=false;
addRecordbasis(datasourceid );
</code>
*existing documents can not be updated in SOLR in our case (only if everything was "stored"). So first, delete the old data from the core. Depending on how many records there are, you might have to delete the data for a specific repository only solrServer:solrPort/solr/coreName_SEARCH_OR_PREVIEW_OR_DETAILS/update?stream.body=<delete><query>YOUR_LIMITATION_FIELD:*</query></delete>&commit=true This "limitation_field" must be an indexed value in the core. If you don't have a lot of data, you can put <query>*:*</query> as the generation of the new SOLR index will be fast enough (5-10min per Core for 180.000 records).
*per default, it will index this new field for each datasource. If it's still missing at the end for some datasources, or if you only want to do it for specific datasources, edit the '''main''' method <code>for (int i=0;i<166;i++)</code>

==Adding them in the portal==
*Edit the SearchController file (controllers)(additions can be made in the CommonSearchController, deletion only in the specific SearchController if you have several portals running in parallel with the same installation)
**if it's a dropdownlist, add the name of the field in the variable <code>$knownLParams</code>.
**if it's a simple term or a suggestion list, add the name of the field in the variable <code>$knownParams</code>.
*Edit the SearchForm file (models)(same rule as above), edit the variable <code>$_parameters</code> - it can be a text, a select(dropdownlist), a suggest, a radio or a range. Also add the public variable with the same name (ie. <code>public $sampleavailability;</code> and
<code>
'sampleavailability' => [
'type' => 'radio',
'name' => 'Sample availability (loan)',
'config' => [
'size' => 285,
'maxlength' => 150,
'name' => 'sampleavailability'
],
'hideable' => 'true'
]
</code>. It will be automatically displayed in the search form. The '''hideable''' parameter specifies if it is displayed per default or if it is only in the accordion.
Eventually, edit the '''attribteLabels''' function such as <code>'sampleavailability' => 'Sample availability (loan)'</code>
*Edit the listView.php (views) file if it has to be displayed here, or the recordViewTabs.php if it has to be displayed on the details page. Here again, add it to the $suggestFields or $defaultFields or $selectFields or $buttonFields, depending of its type.
*Edit the file CreateFormManager(vendor) only if it's a radio button (look how it's done for cites and sampleavailability).
*Edit the file SOLRQueryManager(vendor) - for example, if it has to look in the SOLR case-unsensitive value, do as for
<code>
case 'fullScientificName' :
return $field . '_nc:' . $value . '';
break;
</code> If it's case sensitive,
<code>
case 'collectioncode' :
return $field . ':"' . $value . '"';
break;
</code>

=Running multiple GPS portals in parallel=

*in common/config/bootstrap: the name of all folders that Yii2 should go through (for imports and so)
*in environnments/index.php: the name of the folders that will be exposed and therefore where Yii should be authorized to write logs and handle the CSS files

[[File:gps_folders.jpg]]

=GGBN specific configuration=

The institution has to be filled in bio_datasource (city and institution, for the harvester_factoy like '%Harvester%') - but first the datasource has to be added (metadata update). It will be used for the repository/registry field in the GGBN portal.
The parentInstitution table is used for the contacts.

=Debug=
If something does not work as expected, have a look in the catalina.out file (generated by Tomcat) on the machine where SOLR is running. You will see the last queries sent to SOLR at the end of the file.
The YII file runtime/log/app.log might also help, and is located in the yii folder on the machine running the portal (ie. advanced/runtime....).

Main Page

2016-02-03T14:19:05Z

GabiDroege: /* Workflow */ workflow description added

'''Welcome to the Wiki of the GGBN Portal Software'''

Further information about the software will follow soon.

=Introduction=
The GGBN Portal Software (GPS) has been developed by the Botanic Garden and Botanical Museum Berlin-Dahlem for the Global Genome Biodiversity Network ([http://www.ggbn.org GGBN]). The project was funded by the German Research Foundation (DFG (01/2014 - 03/2016)). The software is written in php using the Yii framework and fully open source. The GPS allows costumization and therefore can be used for other special interest networks too. It is optimized for an underlying SOLR instance, data are harvested with [http://wiki.bgbm.org/bhit/ B-HIT].

=Workflow=
[[File:Workflow GPS.jpg|500px]]

Data are provided through BioCASe and IPT. The harvester [http://wiki.bgbm.org/bhit B-HIT] stores all original data in a MySQL database. To find out more about B-HIT please check out the [http://wiki.bgbm.org/bhit B-HIT Wiki]. After harvesting data are cleaned and can be matched against different taxonomic backbones. A SOLR instance is created on top of the MySQL database. The GPS itself requires access to the SOLR index as well as the MySQL index.

=Installation=

==You will need==
* Apache (tested with Apache/2.4.7 (Ubuntu), Apache/2.2.22 (Debian), Apache/2.4.10 (Win64)) with libapache2-mod-php5
* PHP5 (tested with 5.5.9, 5.5.30 and 5.5.15). You also might need to install php5-xsl, php5-gd and php5-mysql.
* a SOLR instance (tested with Apache SOLR 4.9.0). Contact us for getting the appropriate SOLR configuration files.
* a MySQL database (tested with MySQL 5.5.46). It will contain data harvested thanks to [http://wiki.bgbm.org/bhit/index.php/Main_Page B-HIT]

Yii (version 2) will be fully contained in the subversion code of the portal software.

==Copy a working version from the svn ==
Create a destination folder, for the rest of this documentation we will choose "my_portal" in your web-folder (ie. /var/www/, or *WINPATH*). 
Get http://ww2.biocase.org/svn/dnabank/Dnabank_Portal/ggbn_portal/trunk/ into "my_portal".

=Configuration=
''TBD''
''yii, Apache, rights, SOLR (SOLR config files also & Java)'' 
First, let say we want to access the portal under the URL http://localhost/ggbn_portal.

*Check that the Apache User has writing/editing rights in the whole "my_portal" folder.
*Edit the file "my_portal/frontend/config/main.php".
**Configure the portal URL: <code>$baseUrl = str_replace ( '/SUBDIR_AFTER_YOUR_WEB_FOLDER/my_portal/frontend/web', '/ggbn_portal', (new Request ())->getBaseUrl () );</code>
**Configure the MySQL database credentials <code>
'db' => [
'class' => '\yii\db\Connection',
'dsn' => 'mysql:host=IP;dbname=DBNAME',
'username' => 'USER',
'password' => 'PASS',
'charset' => 'utf8'
],</code>
*Edit the file "my_portal/frontend/config/params.php".
**e-mails
***Configure the admin e-mail <code>'adminEmail' => 'ADMIN@DOMAIN',</code>. The admin will get error/bug messages.
***Configure the support e-mail(s) <code>'supportEmail' => ['MAIL1','MAIL2'..],</code>.
***Configure the feedback e-mail(s) <code>'feedbackMail' => ['MAIL1','MAIL2'..],</code>. They will get the messages sent with the feedback function.
***Configure the no-reply-mail (who is supposed to sent e-mail(s) to the users) <code>'noreplyMail'=>'MAIL'
**Configure the location of the DarwinCore Archives (harvested with B-HIT) <code>'dwcArchivesPath' => 'PATH_TO_THE_PARENT_FOLDER' ,</code>. Check the access rights! (read is enough)
**Configure the location(root) of your SOLR instance <code>'solrhost'=> 'SOLR_HOST'</code> and the SOLR core names
<code>
$searchname="SEARCH_CORE";
$previewname="PREVIEW_CORE";
$detailsname="DETAILS_CORE";
</code>
**Set the temporary folder <code>'tmpFolder'=>'TEMP_FOLDER'</code>. Check the access rights! (write is needed)
**Configure the options
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'noBack'=>true_OR_false</code>: a special backbone is used
***<code>'shopping'=>true_OR_false</code>: the shopping system is enabled or disabled
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'newsService'=>true_OR_false</code>: uses the GGBN news services - if it has to be used with a different wiki news, edit the SiteController.py
***<code>'displayMore'=>true_OR_false</code>: displays the categories with internal links on the details page
***<code>'displayExternals'=>true_OR_false</code>: displays the categories with external links (GBIF, BOLD, NCBI...) on the details page
***<code>'ABCDdownloable'=>true_OR_false</code>: ABCD files are downloadable or not
***<code>'useAnnosys'=>true_OR_false</code>: link to annosys (annotation system) on the details page
***<code>'annosysAP'=>'https://annosys.bgbm.fu-berlin.de/AnnoSys/AnnoSys?providerURL='</code>: root URL for Annosys
***<code>'siteName'=>'my_portal',</code>: the portal name (used in the URL)
***<code>'siteTitle'=>'GGBN Portal',</code>: the default site name/title
***<code>'catalogNumberClickable'=>true,</code>: if the catalogNumber/UnitID can be clickable on the details page

=Personalisation=
If you want to change the icon (favicon), it is located in frontend/web. 
If you want to change the slider images on the home page, they are located in frontend/web/images/slider. Then you will have to modify the view in frontend/view/site/index.php to load the pictures you chose. 
If you want to change the menu icons, they are located in frontend/web/images/icons. 
 
The CSS and JS files are also in the frontend/web folder.

=Adding new search fields or fields to display=
==Adding them in SOLR==
*edit the schema.xml file in SOLR and add the new fields you want to have. If you want to search for them, edit the file for the search core *_search/conf/schema.xml. If wou want to display them in the preview, edit the file for the preview core *_preview/conf/schema.xml. If you need it for the details page, edit the file for the details core *_details/conf/schema.xml. Generally: if you want to search for it (search core), the field has to be indexed, but not mandatory stored. If you want to display it (preview/details), it has to be stored but not indexed. The tripleidstoreid or the occurrenceid *always* has to be indexed.
*restart Tomcat
'''Important ''': always restart Tomcat (service tomcat8 restart) when you modify the schema.xml file.
*edit the Java file corresponding to the core that need to be updated- for example, if you want to index or store the recordbasis, add the following method
<code>
private void addRecordbasis(int datasourceid) throws ClassNotFoundException, SQLException {
System.out.println("RUN RECORDBASIS ");
if(tripleidstoreids.size()>0){
String queryTID =
"SELECT distinct fk_tripleidstoreid as tripleidstoreid, "
+ "IFNULL(recordbasis,'N/A') as recordbasis "
+ " FROM recordbasis "
+ " JOIN occurrence ON recordbasisid = fk_recordbasisid "
+ " WHERE fk_tripleidstoreid in ("+StringUtils.join(tripleidstoreids,", ")+") "
+ " AND fk_datasourceid ="+datasourceid
+ " ORDER BY fk_tripleidstoreid ";
update( queryTID,tripleidstoreids.size());
}
System.gc();
}
</code> and add a call to that method in the '''handleDocuments''' method:
<code>
occurrencesDone.clear();
multivalueForbidden=false;
addRecordbasis(datasourceid );
</code>
*existing documents can not be updated in SOLR in our case (only if everything was "stored"). So first, delete the old data from the core. Depending on how many records there are, you might have to delete the data for a specific repository only solrServer:solrPort/solr/coreName_SEARCH_OR_PREVIEW_OR_DETAILS/update?stream.body=<delete><query>YOUR_LIMITATION_FIELD:*</query></delete>&commit=true This "limitation_field" must be an indexed value in the core. If you don't have a lot of data, you can put <query>*:*</query> as the generation of the new SOLR index will be fast enough (5-10min per Core for 180.000 records).
*per default, it will index this new field for each datasource. If it's still missing at the end for some datasources, or if you only want to do it for specific datasources, edit the '''main''' method <code>for (int i=0;i<166;i++)</code>

==Adding them in the portal==
*Edit the SearchController file (controllers)(additions can be made in the CommonSearchController, deletion only in the specific SearchController if you have several portals running in parallel with the same installation)
**if it's a dropdownlist, add the name of the field in the variable <code>$knownLParams</code>.
**if it's a simple term or a suggestion list, add the name of the field in the variable <code>$knownParams</code>.
*Edit the SearchForm file (models)(same rule as above), edit the variable <code>$_parameters</code> - it can be a text, a select(dropdownlist), a suggest, a radio or a range. Also add the public variable with the same name (ie. <code>public $sampleavailability;</code> and
<code>
'sampleavailability' => [
'type' => 'radio',
'name' => 'Sample availability (loan)',
'config' => [
'size' => 285,
'maxlength' => 150,
'name' => 'sampleavailability'
],
'hideable' => 'true'
]
</code>. It will be automatically displayed in the search form. The '''hideable''' parameter specifies if it is displayed per default or if it is only in the accordion.
Eventually, edit the '''attribteLabels''' function such as <code>'sampleavailability' => 'Sample availability (loan)'</code>
*Edit the listView.php (views) file if it has to be displayed here, or the recordViewTabs.php if it has to be displayed on the details page. Here again, add it to the $suggestFields or $defaultFields or $selectFields or $buttonFields, depending of its type.
*Edit the file CreateFormManager(vendor) only if it's a radio button (look how it's done for cites and sampleavailability).
*Edit the file SOLRQueryManager(vendor) - for example, if it has to look in the SOLR case-unsensitive value, do as for
<code>
case 'fullScientificName' :
return $field . '_nc:' . $value . '';
break;
</code> If it's case sensitive,
<code>
case 'collectioncode' :
return $field . ':"' . $value . '"';
break;
</code>

=Running multiple GPS portals in parallel=

*in common/config/bootstrap: the name of all folders that Yii2 should go through (for imports and so)
*in environnments/index.php: the name of the folders that will be exposed and therefore where Yii should be authorized to write logs and handle the CSS files

[[File:gps_folders.jpg]]

=GGBN specific configuration=

The institution has to be filled in bio_datasource (city and institution, for the harvester_factoy like '%Harvester%') - but first the datasource has to be added (metadata update). It will be used for the repository/registry field in the GGBN portal.
The parentInstitution table is used for the contacts.

=Debug=
If something does not work as expected, have a look in the catalina.out file (generated by Tomcat) on the machine where SOLR is running. You will see the last queries sent to SOLR at the end of the file.
The YII file runtime/log/app.log might also help, and is located in the yii folder on the machine running the portal (ie. advanced/runtime....).

Main Page

2016-02-03T14:09:49Z

GabiDroege: /* Introduction */

'''Welcome to the Wiki of the GGBN Portal Software'''

Further information about the software will follow soon.

=Introduction=
The GGBN Portal Software (GPS) has been developed by the Botanic Garden and Botanical Museum Berlin-Dahlem for the Global Genome Biodiversity Network ([http://www.ggbn.org GGBN]). The project was funded by the German Research Foundation (DFG (01/2014 - 03/2016)). The software is written in php using the Yii framework and fully open source. The GPS allows costumization and therefore can be used for other special interest networks too. It is optimized for an underlying SOLR instance, data are harvested with [http://wiki.bgbm.org/bhit/ B-HIT].

=Workflow=
[[File:Workflow GPS.jpg|500px]]

=Installation=

==You will need==
* Apache (tested with Apache/2.4.7 (Ubuntu), Apache/2.2.22 (Debian), Apache/2.4.10 (Win64)) with libapache2-mod-php5
* PHP5 (tested with 5.5.9, 5.5.30 and 5.5.15). You also might need to install php5-xsl, php5-gd and php5-mysql.
* a SOLR instance (tested with Apache SOLR 4.9.0). Contact us for getting the appropriate SOLR configuration files.
* a MySQL database (tested with MySQL 5.5.46). It will contain data harvested thanks to [http://wiki.bgbm.org/bhit/index.php/Main_Page B-HIT]

Yii (version 2) will be fully contained in the subversion code of the portal software.

==Copy a working version from the svn ==
Create a destination folder, for the rest of this documentation we will choose "my_portal" in your web-folder (ie. /var/www/, or *WINPATH*). 
Get http://ww2.biocase.org/svn/dnabank/Dnabank_Portal/ggbn_portal/trunk/ into "my_portal".

=Configuration=
''TBD''
''yii, Apache, rights, SOLR (SOLR config files also & Java)'' 
First, let say we want to access the portal under the URL http://localhost/ggbn_portal.

*Check that the Apache User has writing/editing rights in the whole "my_portal" folder.
*Edit the file "my_portal/frontend/config/main.php".
**Configure the portal URL: <code>$baseUrl = str_replace ( '/SUBDIR_AFTER_YOUR_WEB_FOLDER/my_portal/frontend/web', '/ggbn_portal', (new Request ())->getBaseUrl () );</code>
**Configure the MySQL database credentials <code>
'db' => [
'class' => '\yii\db\Connection',
'dsn' => 'mysql:host=IP;dbname=DBNAME',
'username' => 'USER',
'password' => 'PASS',
'charset' => 'utf8'
],</code>
*Edit the file "my_portal/frontend/config/params.php".
**e-mails
***Configure the admin e-mail <code>'adminEmail' => 'ADMIN@DOMAIN',</code>. The admin will get error/bug messages.
***Configure the support e-mail(s) <code>'supportEmail' => ['MAIL1','MAIL2'..],</code>.
***Configure the feedback e-mail(s) <code>'feedbackMail' => ['MAIL1','MAIL2'..],</code>. They will get the messages sent with the feedback function.
***Configure the no-reply-mail (who is supposed to sent e-mail(s) to the users) <code>'noreplyMail'=>'MAIL'
**Configure the location of the DarwinCore Archives (harvested with B-HIT) <code>'dwcArchivesPath' => 'PATH_TO_THE_PARENT_FOLDER' ,</code>. Check the access rights! (read is enough)
**Configure the location(root) of your SOLR instance <code>'solrhost'=> 'SOLR_HOST'</code> and the SOLR core names
<code>
$searchname="SEARCH_CORE";
$previewname="PREVIEW_CORE";
$detailsname="DETAILS_CORE";
</code>
**Set the temporary folder <code>'tmpFolder'=>'TEMP_FOLDER'</code>. Check the access rights! (write is needed)
**Configure the options
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'noBack'=>true_OR_false</code>: a special backbone is used
***<code>'shopping'=>true_OR_false</code>: the shopping system is enabled or disabled
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'newsService'=>true_OR_false</code>: uses the GGBN news services - if it has to be used with a different wiki news, edit the SiteController.py
***<code>'displayMore'=>true_OR_false</code>: displays the categories with internal links on the details page
***<code>'displayExternals'=>true_OR_false</code>: displays the categories with external links (GBIF, BOLD, NCBI...) on the details page
***<code>'ABCDdownloable'=>true_OR_false</code>: ABCD files are downloadable or not
***<code>'useAnnosys'=>true_OR_false</code>: link to annosys (annotation system) on the details page
***<code>'annosysAP'=>'https://annosys.bgbm.fu-berlin.de/AnnoSys/AnnoSys?providerURL='</code>: root URL for Annosys
***<code>'siteName'=>'my_portal',</code>: the portal name (used in the URL)
***<code>'siteTitle'=>'GGBN Portal',</code>: the default site name/title
***<code>'catalogNumberClickable'=>true,</code>: if the catalogNumber/UnitID can be clickable on the details page

=Personalisation=
If you want to change the icon (favicon), it is located in frontend/web. 
If you want to change the slider images on the home page, they are located in frontend/web/images/slider. Then you will have to modify the view in frontend/view/site/index.php to load the pictures you chose. 
If you want to change the menu icons, they are located in frontend/web/images/icons. 
 
The CSS and JS files are also in the frontend/web folder.

=Adding new search fields or fields to display=
==Adding them in SOLR==
*edit the schema.xml file in SOLR and add the new fields you want to have. If you want to search for them, edit the file for the search core *_search/conf/schema.xml. If wou want to display them in the preview, edit the file for the preview core *_preview/conf/schema.xml. If you need it for the details page, edit the file for the details core *_details/conf/schema.xml. Generally: if you want to search for it (search core), the field has to be indexed, but not mandatory stored. If you want to display it (preview/details), it has to be stored but not indexed. The tripleidstoreid or the occurrenceid *always* has to be indexed.
*restart Tomcat
'''Important ''': always restart Tomcat (service tomcat8 restart) when you modify the schema.xml file.
*edit the Java file corresponding to the core that need to be updated- for example, if you want to index or store the recordbasis, add the following method
<code>
private void addRecordbasis(int datasourceid) throws ClassNotFoundException, SQLException {
System.out.println("RUN RECORDBASIS ");
if(tripleidstoreids.size()>0){
String queryTID =
"SELECT distinct fk_tripleidstoreid as tripleidstoreid, "
+ "IFNULL(recordbasis,'N/A') as recordbasis "
+ " FROM recordbasis "
+ " JOIN occurrence ON recordbasisid = fk_recordbasisid "
+ " WHERE fk_tripleidstoreid in ("+StringUtils.join(tripleidstoreids,", ")+") "
+ " AND fk_datasourceid ="+datasourceid
+ " ORDER BY fk_tripleidstoreid ";
update( queryTID,tripleidstoreids.size());
}
System.gc();
}
</code> and add a call to that method in the '''handleDocuments''' method:
<code>
occurrencesDone.clear();
multivalueForbidden=false;
addRecordbasis(datasourceid );
</code>
*existing documents can not be updated in SOLR in our case (only if everything was "stored"). So first, delete the old data from the core. Depending on how many records there are, you might have to delete the data for a specific repository only solrServer:solrPort/solr/coreName_SEARCH_OR_PREVIEW_OR_DETAILS/update?stream.body=<delete><query>YOUR_LIMITATION_FIELD:*</query></delete>&commit=true This "limitation_field" must be an indexed value in the core. If you don't have a lot of data, you can put <query>*:*</query> as the generation of the new SOLR index will be fast enough (5-10min per Core for 180.000 records).
*per default, it will index this new field for each datasource. If it's still missing at the end for some datasources, or if you only want to do it for specific datasources, edit the '''main''' method <code>for (int i=0;i<166;i++)</code>

==Adding them in the portal==
*Edit the SearchController file (controllers)(additions can be made in the CommonSearchController, deletion only in the specific SearchController if you have several portals running in parallel with the same installation)
**if it's a dropdownlist, add the name of the field in the variable <code>$knownLParams</code>.
**if it's a simple term or a suggestion list, add the name of the field in the variable <code>$knownParams</code>.
*Edit the SearchForm file (models)(same rule as above), edit the variable <code>$_parameters</code> - it can be a text, a select(dropdownlist), a suggest, a radio or a range. Also add the public variable with the same name (ie. <code>public $sampleavailability;</code> and
<code>
'sampleavailability' => [
'type' => 'radio',
'name' => 'Sample availability (loan)',
'config' => [
'size' => 285,
'maxlength' => 150,
'name' => 'sampleavailability'
],
'hideable' => 'true'
]
</code>. It will be automatically displayed in the search form. The '''hideable''' parameter specifies if it is displayed per default or if it is only in the accordion.
Eventually, edit the '''attribteLabels''' function such as <code>'sampleavailability' => 'Sample availability (loan)'</code>
*Edit the listView.php (views) file if it has to be displayed here, or the recordViewTabs.php if it has to be displayed on the details page. Here again, add it to the $suggestFields or $defaultFields or $selectFields or $buttonFields, depending of its type.
*Edit the file CreateFormManager(vendor) only if it's a radio button (look how it's done for cites and sampleavailability).
*Edit the file SOLRQueryManager(vendor) - for example, if it has to look in the SOLR case-unsensitive value, do as for
<code>
case 'fullScientificName' :
return $field . '_nc:' . $value . '';
break;
</code> If it's case sensitive,
<code>
case 'collectioncode' :
return $field . ':"' . $value . '"';
break;
</code>

=Running multiple GPS portals in parallel=

*in common/config/bootstrap: the name of all folders that Yii2 should go through (for imports and so)
*in environnments/index.php: the name of the folders that will be exposed and therefore where Yii should be authorized to write logs and handle the CSS files

[[File:gps_folders.jpg]]

=GGBN specific configuration=

The institution has to be filled in bio_datasource (city and institution, for the harvester_factoy like '%Harvester%') - but first the datasource has to be added (metadata update). It will be used for the repository/registry field in the GGBN portal.
The parentInstitution table is used for the contacts.

=Debug=
If something does not work as expected, have a look in the catalina.out file (generated by Tomcat) on the machine where SOLR is running. You will see the last queries sent to SOLR at the end of the file.
The YII file runtime/log/app.log might also help, and is located in the yii folder on the machine running the portal (ie. advanced/runtime....).

Main Page

2016-02-03T14:09:35Z

GabiDroege: /* Introduction */

'''Welcome to the Wiki of the GGBN Portal Software'''

Further information about the software will follow soon.

=Introduction=
The GGBN Portal Software (GPS) has been developed by the Botanic Garden and Botanical Museum Berlin-Dahlem for the Global Genome Biodiversity Network ([http://www.ggbn.org | GGBN]). The project was funded by the German Research Foundation (DFG (01/2014 - 03/2016)). The software is written in php using the Yii framework and fully open source. The GPS allows costumization and therefore can be used for other special interest networks too. It is optimized for an underlying SOLR instance, data are harvested with [http://wiki.bgbm.org/bhit/ | B-HIT].

=Workflow=
[[File:Workflow GPS.jpg|500px]]

=Installation=

==You will need==
* Apache (tested with Apache/2.4.7 (Ubuntu), Apache/2.2.22 (Debian), Apache/2.4.10 (Win64)) with libapache2-mod-php5
* PHP5 (tested with 5.5.9, 5.5.30 and 5.5.15). You also might need to install php5-xsl, php5-gd and php5-mysql.
* a SOLR instance (tested with Apache SOLR 4.9.0). Contact us for getting the appropriate SOLR configuration files.
* a MySQL database (tested with MySQL 5.5.46). It will contain data harvested thanks to [http://wiki.bgbm.org/bhit/index.php/Main_Page B-HIT]

Yii (version 2) will be fully contained in the subversion code of the portal software.

==Copy a working version from the svn ==
Create a destination folder, for the rest of this documentation we will choose "my_portal" in your web-folder (ie. /var/www/, or *WINPATH*). 
Get http://ww2.biocase.org/svn/dnabank/Dnabank_Portal/ggbn_portal/trunk/ into "my_portal".

=Configuration=
''TBD''
''yii, Apache, rights, SOLR (SOLR config files also & Java)'' 
First, let say we want to access the portal under the URL http://localhost/ggbn_portal.

*Check that the Apache User has writing/editing rights in the whole "my_portal" folder.
*Edit the file "my_portal/frontend/config/main.php".
**Configure the portal URL: <code>$baseUrl = str_replace ( '/SUBDIR_AFTER_YOUR_WEB_FOLDER/my_portal/frontend/web', '/ggbn_portal', (new Request ())->getBaseUrl () );</code>
**Configure the MySQL database credentials <code>
'db' => [
'class' => '\yii\db\Connection',
'dsn' => 'mysql:host=IP;dbname=DBNAME',
'username' => 'USER',
'password' => 'PASS',
'charset' => 'utf8'
],</code>
*Edit the file "my_portal/frontend/config/params.php".
**e-mails
***Configure the admin e-mail <code>'adminEmail' => 'ADMIN@DOMAIN',</code>. The admin will get error/bug messages.
***Configure the support e-mail(s) <code>'supportEmail' => ['MAIL1','MAIL2'..],</code>.
***Configure the feedback e-mail(s) <code>'feedbackMail' => ['MAIL1','MAIL2'..],</code>. They will get the messages sent with the feedback function.
***Configure the no-reply-mail (who is supposed to sent e-mail(s) to the users) <code>'noreplyMail'=>'MAIL'
**Configure the location of the DarwinCore Archives (harvested with B-HIT) <code>'dwcArchivesPath' => 'PATH_TO_THE_PARENT_FOLDER' ,</code>. Check the access rights! (read is enough)
**Configure the location(root) of your SOLR instance <code>'solrhost'=> 'SOLR_HOST'</code> and the SOLR core names
<code>
$searchname="SEARCH_CORE";
$previewname="PREVIEW_CORE";
$detailsname="DETAILS_CORE";
</code>
**Set the temporary folder <code>'tmpFolder'=>'TEMP_FOLDER'</code>. Check the access rights! (write is needed)
**Configure the options
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'noBack'=>true_OR_false</code>: a special backbone is used
***<code>'shopping'=>true_OR_false</code>: the shopping system is enabled or disabled
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'newsService'=>true_OR_false</code>: uses the GGBN news services - if it has to be used with a different wiki news, edit the SiteController.py
***<code>'displayMore'=>true_OR_false</code>: displays the categories with internal links on the details page
***<code>'displayExternals'=>true_OR_false</code>: displays the categories with external links (GBIF, BOLD, NCBI...) on the details page
***<code>'ABCDdownloable'=>true_OR_false</code>: ABCD files are downloadable or not
***<code>'useAnnosys'=>true_OR_false</code>: link to annosys (annotation system) on the details page
***<code>'annosysAP'=>'https://annosys.bgbm.fu-berlin.de/AnnoSys/AnnoSys?providerURL='</code>: root URL for Annosys
***<code>'siteName'=>'my_portal',</code>: the portal name (used in the URL)
***<code>'siteTitle'=>'GGBN Portal',</code>: the default site name/title
***<code>'catalogNumberClickable'=>true,</code>: if the catalogNumber/UnitID can be clickable on the details page

=Personalisation=
If you want to change the icon (favicon), it is located in frontend/web. 
If you want to change the slider images on the home page, they are located in frontend/web/images/slider. Then you will have to modify the view in frontend/view/site/index.php to load the pictures you chose. 
If you want to change the menu icons, they are located in frontend/web/images/icons. 
 
The CSS and JS files are also in the frontend/web folder.

=Adding new search fields or fields to display=
==Adding them in SOLR==
*edit the schema.xml file in SOLR and add the new fields you want to have. If you want to search for them, edit the file for the search core *_search/conf/schema.xml. If wou want to display them in the preview, edit the file for the preview core *_preview/conf/schema.xml. If you need it for the details page, edit the file for the details core *_details/conf/schema.xml. Generally: if you want to search for it (search core), the field has to be indexed, but not mandatory stored. If you want to display it (preview/details), it has to be stored but not indexed. The tripleidstoreid or the occurrenceid *always* has to be indexed.
*restart Tomcat
'''Important ''': always restart Tomcat (service tomcat8 restart) when you modify the schema.xml file.
*edit the Java file corresponding to the core that need to be updated- for example, if you want to index or store the recordbasis, add the following method
<code>
private void addRecordbasis(int datasourceid) throws ClassNotFoundException, SQLException {
System.out.println("RUN RECORDBASIS ");
if(tripleidstoreids.size()>0){
String queryTID =
"SELECT distinct fk_tripleidstoreid as tripleidstoreid, "
+ "IFNULL(recordbasis,'N/A') as recordbasis "
+ " FROM recordbasis "
+ " JOIN occurrence ON recordbasisid = fk_recordbasisid "
+ " WHERE fk_tripleidstoreid in ("+StringUtils.join(tripleidstoreids,", ")+") "
+ " AND fk_datasourceid ="+datasourceid
+ " ORDER BY fk_tripleidstoreid ";
update( queryTID,tripleidstoreids.size());
}
System.gc();
}
</code> and add a call to that method in the '''handleDocuments''' method:
<code>
occurrencesDone.clear();
multivalueForbidden=false;
addRecordbasis(datasourceid );
</code>
*existing documents can not be updated in SOLR in our case (only if everything was "stored"). So first, delete the old data from the core. Depending on how many records there are, you might have to delete the data for a specific repository only solrServer:solrPort/solr/coreName_SEARCH_OR_PREVIEW_OR_DETAILS/update?stream.body=<delete><query>YOUR_LIMITATION_FIELD:*</query></delete>&commit=true This "limitation_field" must be an indexed value in the core. If you don't have a lot of data, you can put <query>*:*</query> as the generation of the new SOLR index will be fast enough (5-10min per Core for 180.000 records).
*per default, it will index this new field for each datasource. If it's still missing at the end for some datasources, or if you only want to do it for specific datasources, edit the '''main''' method <code>for (int i=0;i<166;i++)</code>

==Adding them in the portal==
*Edit the SearchController file (controllers)(additions can be made in the CommonSearchController, deletion only in the specific SearchController if you have several portals running in parallel with the same installation)
**if it's a dropdownlist, add the name of the field in the variable <code>$knownLParams</code>.
**if it's a simple term or a suggestion list, add the name of the field in the variable <code>$knownParams</code>.
*Edit the SearchForm file (models)(same rule as above), edit the variable <code>$_parameters</code> - it can be a text, a select(dropdownlist), a suggest, a radio or a range. Also add the public variable with the same name (ie. <code>public $sampleavailability;</code> and
<code>
'sampleavailability' => [
'type' => 'radio',
'name' => 'Sample availability (loan)',
'config' => [
'size' => 285,
'maxlength' => 150,
'name' => 'sampleavailability'
],
'hideable' => 'true'
]
</code>. It will be automatically displayed in the search form. The '''hideable''' parameter specifies if it is displayed per default or if it is only in the accordion.
Eventually, edit the '''attribteLabels''' function such as <code>'sampleavailability' => 'Sample availability (loan)'</code>
*Edit the listView.php (views) file if it has to be displayed here, or the recordViewTabs.php if it has to be displayed on the details page. Here again, add it to the $suggestFields or $defaultFields or $selectFields or $buttonFields, depending of its type.
*Edit the file CreateFormManager(vendor) only if it's a radio button (look how it's done for cites and sampleavailability).
*Edit the file SOLRQueryManager(vendor) - for example, if it has to look in the SOLR case-unsensitive value, do as for
<code>
case 'fullScientificName' :
return $field . '_nc:' . $value . '';
break;
</code> If it's case sensitive,
<code>
case 'collectioncode' :
return $field . ':"' . $value . '"';
break;
</code>

=Running multiple GPS portals in parallel=

*in common/config/bootstrap: the name of all folders that Yii2 should go through (for imports and so)
*in environnments/index.php: the name of the folders that will be exposed and therefore where Yii should be authorized to write logs and handle the CSS files

[[File:gps_folders.jpg]]

=GGBN specific configuration=

The institution has to be filled in bio_datasource (city and institution, for the harvester_factoy like '%Harvester%') - but first the datasource has to be added (metadata update). It will be used for the repository/registry field in the GGBN portal.
The parentInstitution table is used for the contacts.

=Debug=
If something does not work as expected, have a look in the catalina.out file (generated by Tomcat) on the machine where SOLR is running. You will see the last queries sent to SOLR at the end of the file.
The YII file runtime/log/app.log might also help, and is located in the yii folder on the machine running the portal (ie. advanced/runtime....).

MediaWiki:Sidebar

2016-02-03T13:08:30Z

GabiDroege:

* SEARCH
*Home
**Main_Page | GPS Wiki

*Resources
**http://www.ggbn.org | GGBN Home
**http://wiki.bgbm.org/bps/ | B-HIT Wiki

* TOOLBOX

Main Page

2016-02-03T13:06:24Z

GabiDroege: /* Workflow */ added figure

'''Welcome to the Wiki of the GGBN Portal Software'''

Further information about the software will follow soon.

=Introduction=
The GGBN Portal Software (GPS) has been developed by the Botanic Garden and Botanical Museum Berlin-Dahlem for the Global Genome Biodiversity Network (GGBN). The project was funded by the German Research Foundation (DFG (01/2014 - 03/2016). The software is written in php using the Yii framework and fully open source. The GPS allows costumization and therefore can be used for other special interest networks too. It is optimized for an underlying SOLR instance, data are harvested with B-HIT.

=Workflow=
[[File:Workflow GPS.jpg|500px]]

=Installation=

==You will need==
* Apache (tested with Apache/2.4.7 (Ubuntu), Apache/2.2.22 (Debian), Apache/2.4.10 (Win64)) with libapache2-mod-php5
* PHP5 (tested with 5.5.9, 5.5.30 and 5.5.15). You also might need to install php5-xsl, php5-gd and php5-mysql.
* a SOLR instance (tested with Apache SOLR 4.9.0). Contact us for getting the appropriate SOLR configuration files.
* a MySQL database (tested with MySQL 5.5.46). It will contain data harvested thanks to [http://wiki.bgbm.org/bhit/index.php/Main_Page B-HIT]

Yii (version 2) will be fully contained in the subversion code of the portal software.

==Copy a working version from the svn ==
Create a destination folder, for the rest of this documentation we will choose "my_portal" in your web-folder (ie. /var/www/, or *WINPATH*). 
Get http://ww2.biocase.org/svn/dnabank/Dnabank_Portal/ggbn_portal/trunk/ into "my_portal".

=Configuration=
''TBD''
''yii, Apache, rights, SOLR (SOLR config files also & Java)'' 
First, let say we want to access the portal under the URL http://localhost/ggbn_portal.

*Check that the Apache User has writing/editing rights in the whole "my_portal" folder.
*Edit the file "my_portal/frontend/config/main.php".
**Configure the portal URL: <code>$baseUrl = str_replace ( '/SUBDIR_AFTER_YOUR_WEB_FOLDER/my_portal/frontend/web', '/ggbn_portal', (new Request ())->getBaseUrl () );</code>
**Configure the MySQL database credentials <code>
'db' => [
'class' => '\yii\db\Connection',
'dsn' => 'mysql:host=IP;dbname=DBNAME',
'username' => 'USER',
'password' => 'PASS',
'charset' => 'utf8'
],</code>
*Edit the file "my_portal/frontend/config/params.php".
**e-mails
***Configure the admin e-mail <code>'adminEmail' => 'ADMIN@DOMAIN',</code>. The admin will get error/bug messages.
***Configure the support e-mail(s) <code>'supportEmail' => ['MAIL1','MAIL2'..],</code>.
***Configure the feedback e-mail(s) <code>'feedbackMail' => ['MAIL1','MAIL2'..],</code>. They will get the messages sent with the feedback function.
***Configure the no-reply-mail (who is supposed to sent e-mail(s) to the users) <code>'noreplyMail'=>'MAIL'
**Configure the location of the DarwinCore Archives (harvested with B-HIT) <code>'dwcArchivesPath' => 'PATH_TO_THE_PARENT_FOLDER' ,</code>. Check the access rights! (read is enough)
**Configure the location(root) of your SOLR instance <code>'solrhost'=> 'SOLR_HOST'</code> and the SOLR core names
<code>
$searchname="SEARCH_CORE";
$previewname="PREVIEW_CORE";
$detailsname="DETAILS_CORE";
</code>
**Set the temporary folder <code>'tmpFolder'=>'TEMP_FOLDER'</code>. Check the access rights! (write is needed)
**Configure the options
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'noBack'=>true_OR_false</code>: a special backbone is used
***<code>'shopping'=>true_OR_false</code>: the shopping system is enabled or disabled
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'newsService'=>true_OR_false</code>: uses the GGBN news services - if it has to be used with a different wiki news, edit the SiteController.py
***<code>'displayMore'=>true_OR_false</code>: displays the categories with internal links on the details page
***<code>'displayExternals'=>true_OR_false</code>: displays the categories with external links (GBIF, BOLD, NCBI...) on the details page
***<code>'ABCDdownloable'=>true_OR_false</code>: ABCD files are downloadable or not
***<code>'useAnnosys'=>true_OR_false</code>: link to annosys (annotation system) on the details page
***<code>'annosysAP'=>'https://annosys.bgbm.fu-berlin.de/AnnoSys/AnnoSys?providerURL='</code>: root URL for Annosys
***<code>'siteName'=>'my_portal',</code>: the portal name (used in the URL)
***<code>'siteTitle'=>'GGBN Portal',</code>: the default site name/title
***<code>'catalogNumberClickable'=>true,</code>: if the catalogNumber/UnitID can be clickable on the details page

=Personalisation=
If you want to change the icon (favicon), it is located in frontend/web. 
If you want to change the slider images on the home page, they are located in frontend/web/images/slider. Then you will have to modify the view in frontend/view/site/index.php to load the pictures you chose. 
If you want to change the menu icons, they are located in frontend/web/images/icons. 
 
The CSS and JS files are also in the frontend/web folder.

=Adding new search fields or fields to display=
==Adding them in SOLR==
*edit the schema.xml file in SOLR and add the new fields you want to have. If you want to search for them, edit the file for the search core *_search/conf/schema.xml. If wou want to display them in the preview, edit the file for the preview core *_preview/conf/schema.xml. If you need it for the details page, edit the file for the details core *_details/conf/schema.xml. Generally: if you want to search for it (search core), the field has to be indexed, but not mandatory stored. If you want to display it (preview/details), it has to be stored but not indexed. The tripleidstoreid or the occurrenceid *always* has to be indexed.
*restart Tomcat
'''Important ''': always restart Tomcat (service tomcat8 restart) when you modify the schema.xml file.
*edit the Java file corresponding to the core that need to be updated- for example, if you want to index or store the recordbasis, add the following method
<code>
private void addRecordbasis(int datasourceid) throws ClassNotFoundException, SQLException {
System.out.println("RUN RECORDBASIS ");
if(tripleidstoreids.size()>0){
String queryTID =
"SELECT distinct fk_tripleidstoreid as tripleidstoreid, "
+ "IFNULL(recordbasis,'N/A') as recordbasis "
+ " FROM recordbasis "
+ " JOIN occurrence ON recordbasisid = fk_recordbasisid "
+ " WHERE fk_tripleidstoreid in ("+StringUtils.join(tripleidstoreids,", ")+") "
+ " AND fk_datasourceid ="+datasourceid
+ " ORDER BY fk_tripleidstoreid ";
update( queryTID,tripleidstoreids.size());
}
System.gc();
}
</code> and add a call to that method in the '''handleDocuments''' method:
<code>
occurrencesDone.clear();
multivalueForbidden=false;
addRecordbasis(datasourceid );
</code>
*existing documents can not be updated in SOLR in our case (only if everything was "stored"). So first, delete the old data from the core. Depending on how many records there are, you might have to delete the data for a specific repository only solrServer:solrPort/solr/coreName_SEARCH_OR_PREVIEW_OR_DETAILS/update?stream.body=<delete><query>YOUR_LIMITATION_FIELD:*</query></delete>&commit=true This "limitation_field" must be an indexed value in the core. If you don't have a lot of data, you can put <query>*:*</query> as the generation of the new SOLR index will be fast enough (5-10min per Core for 180.000 records).
*per default, it will index this new field for each datasource. If it's still missing at the end for some datasources, or if you only want to do it for specific datasources, edit the '''main''' method <code>for (int i=0;i<166;i++)</code>

==Adding them in the portal==
*Edit the SearchController file (controllers)(additions can be made in the CommonSearchController, deletion only in the specific SearchController if you have several portals running in parallel with the same installation)
**if it's a dropdownlist, add the name of the field in the variable <code>$knownLParams</code>.
**if it's a simple term or a suggestion list, add the name of the field in the variable <code>$knownParams</code>.
*Edit the SearchForm file (models)(same rule as above), edit the variable <code>$_parameters</code> - it can be a text, a select(dropdownlist), a suggest, a radio or a range. Also add the public variable with the same name (ie. <code>public $sampleavailability;</code> and
<code>
'sampleavailability' => [
'type' => 'radio',
'name' => 'Sample availability (loan)',
'config' => [
'size' => 285,
'maxlength' => 150,
'name' => 'sampleavailability'
],
'hideable' => 'true'
]
</code>. It will be automatically displayed in the search form. The '''hideable''' parameter specifies if it is displayed per default or if it is only in the accordion.
Eventually, edit the '''attribteLabels''' function such as <code>'sampleavailability' => 'Sample availability (loan)'</code>
*Edit the listView.php (views) file if it has to be displayed here, or the recordViewTabs.php if it has to be displayed on the details page. Here again, add it to the $suggestFields or $defaultFields or $selectFields or $buttonFields, depending of its type.
*Edit the file CreateFormManager(vendor) only if it's a radio button (look how it's done for cites and sampleavailability).
*Edit the file SOLRQueryManager(vendor) - for example, if it has to look in the SOLR case-unsensitive value, do as for
<code>
case 'fullScientificName' :
return $field . '_nc:' . $value . '';
break;
</code> If it's case sensitive,
<code>
case 'collectioncode' :
return $field . ':"' . $value . '"';
break;
</code>

=Running multiple GPS portals in parallel=

*in common/config/bootstrap: the name of all folders that Yii2 should go through (for imports and so)
*in environnments/index.php: the name of the folders that will be exposed and therefore where Yii should be authorized to write logs and handle the CSS files

[[File:gps_folders.jpg]]

=GGBN specific configuration=

The institution has to be filled in bio_datasource (city and institution, for the harvester_factoy like '%Harvester%') - but first the datasource has to be added (metadata update). It will be used for the repository/registry field in the GGBN portal.
The parentInstitution table is used for the contacts.

=Debug=
If something does not work as expected, have a look in the catalina.out file (generated by Tomcat) on the machine where SOLR is running. You will see the last queries sent to SOLR at the end of the file.
The YII file runtime/log/app.log might also help, and is located in the yii folder on the machine running the portal (ie. advanced/runtime....).

File:Workflow GPS.jpg

2016-02-03T13:05:43Z

GabiDroege:

MediaWiki:Common.js

2016-02-03T13:00:29Z

GabiDroege: Created page with "/* Any JavaScript here will be loaded for all users on every page load. */ document.getElementsByTagName("body")[0].classList.remove("vector-animateLayout"); if(skin=="ggbn_g..."

/* Any JavaScript here will be loaded for all users on every page load. */
document.getElementsByTagName("body")[0].classList.remove("vector-animateLayout");

if(skin=="ggbn_gps"){
toolbox = document.getElementById("p-tb");
headmenu = document.getElementById("mw-head");
if(toolbox && headmenu){
toolbox.parentNode.removeChild(toolbox);
toolbox.setAttribute("class","vectorTabs");
headmenu.appendChild(toolbox);
}
portalLinks = document.getElementsByClassName("portal");
for(i=0;i<portalLinks.length;i++){
var portalLink = portalLinks[i];
portalLink.setAttribute('title',portalLink.getAttribute('id').substring(2).replace(new RegExp('_', 'g'), ' '));
}
}

MediaWiki:Sidebar

2016-02-03T12:58:30Z

GabiDroege:

* SEARCH
*Home
**Main_Page | GGBN Wiki
**http://www.ggbn.org | GGBN Home

* TOOLBOX

MediaWiki:Sidebar

2016-02-03T12:51:45Z

GabiDroege: 1 revision: sidebar

* navigation
** mainpage|mainpage-description
** recentchanges-url|recentchanges
** randompage-url|randompage
** helppage|help
* SEARCH
* TOOLBOX
* LANGUAGES

MediaWiki:Sidebar

2016-02-03T12:50:48Z

GabiDroege: Created page with "* navigation ** mainpage|mainpage-description ** recentchanges-url|recentchanges ** randompage-url|randompage ** helppage|help * SEARCH * TOOLBOX * LANGUAGES"

* navigation
** mainpage|mainpage-description
** recentchanges-url|recentchanges
** randompage-url|randompage
** helppage|help
* SEARCH
* TOOLBOX
* LANGUAGES

Main Page

2016-02-03T09:33:30Z

GabiDroege: added introduction, to be continued

'''Welcome to the Wiki of the GGBN Portal Software'''

Further information about the software will follow soon.

=Introduction=
The GGBN Portal Software (GPS) has been developed by the Botanic Garden and Botanical Museum Berlin-Dahlem for the Global Genome Biodiversity Network (GGBN). The project was funded by the German Research Foundation (DFG (01/2014 - 03/2016). The software is written in php using the Yii framework and fully open source. The GPS allows costumization and therefore can be used for other special interest networks too. It is optimized for an underlying SOLR instance, data are harvested with B-HIT.

=Workflow=
TBD

=Installation=

==You will need==
* Apache (tested with Apache/2.4.7 (Ubuntu), Apache/2.2.22 (Debian), Apache/2.4.10 (Win64)) with libapache2-mod-php5
* PHP5 (tested with 5.5.9, 5.5.30 and 5.5.15). You also might need to install php5-xsl, php5-gd and php5-mysql.
* a SOLR instance (tested with Apache SOLR 4.9.0). Contact us for getting the appropriate SOLR configuration files.
* a MySQL database (tested with MySQL 5.5.46). It will contain data harvested thanks to [http://wiki.bgbm.org/bhit/index.php/Main_Page B-HIT]

Yii (version 2) will be fully contained in the subversion code of the portal software.

==Copy a working version from the svn ==
Create a destination folder, for the rest of this documentation we will choose "my_portal" in your web-folder (ie. /var/www/, or *WINPATH*). 
Get http://ww2.biocase.org/svn/dnabank/Dnabank_Portal/ggbn_portal/trunk/ into "my_portal".

=Configuration=
''TBD''
''yii, Apache, rights, SOLR (SOLR config files also & Java)'' 
First, let say we want to access the portal under the URL http://localhost/ggbn_portal.

*Check that the Apache User has writing/editing rights in the whole "my_portal" folder.
*Edit the file "my_portal/frontend/config/main.php".
**Configure the portal URL: <code>$baseUrl = str_replace ( '/SUBDIR_AFTER_YOUR_WEB_FOLDER/my_portal/frontend/web', '/ggbn_portal', (new Request ())->getBaseUrl () );</code>
**Configure the MySQL database credentials <code>
'db' => [
'class' => '\yii\db\Connection',
'dsn' => 'mysql:host=IP;dbname=DBNAME',
'username' => 'USER',
'password' => 'PASS',
'charset' => 'utf8'
],</code>
*Edit the file "my_portal/frontend/config/params.php".
**e-mails
***Configure the admin e-mail <code>'adminEmail' => 'ADMIN@DOMAIN',</code>. The admin will get error/bug messages.
***Configure the support e-mail(s) <code>'supportEmail' => ['MAIL1','MAIL2'..],</code>.
***Configure the feedback e-mail(s) <code>'feedbackMail' => ['MAIL1','MAIL2'..],</code>. They will get the messages sent with the feedback function.
***Configure the no-reply-mail (who is supposed to sent e-mail(s) to the users) <code>'noreplyMail'=>'MAIL'
**Configure the location of the DarwinCore Archives (harvested with B-HIT) <code>'dwcArchivesPath' => 'PATH_TO_THE_PARENT_FOLDER' ,</code>. Check the access rights! (read is enough)
**Configure the location(root) of your SOLR instance <code>'solrhost'=> 'SOLR_HOST'</code> and the SOLR core names
<code>
$searchname="SEARCH_CORE";
$previewname="PREVIEW_CORE";
$detailsname="DETAILS_CORE";
</code>
**Set the temporary folder <code>'tmpFolder'=>'TEMP_FOLDER'</code>. Check the access rights! (write is needed)
**Configure the options
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'noBack'=>true_OR_false</code>: a special backbone is used
***<code>'shopping'=>true_OR_false</code>: the shopping system is enabled or disabled
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'newsService'=>true_OR_false</code>: uses the GGBN news services - if it has to be used with a different wiki news, edit the SiteController.py
***<code>'displayMore'=>true_OR_false</code>: displays the categories with internal links on the details page
***<code>'displayExternals'=>true_OR_false</code>: displays the categories with external links (GBIF, BOLD, NCBI...) on the details page
***<code>'ABCDdownloable'=>true_OR_false</code>: ABCD files are downloadable or not
***<code>'useAnnosys'=>true_OR_false</code>: link to annosys (annotation system) on the details page
***<code>'annosysAP'=>'https://annosys.bgbm.fu-berlin.de/AnnoSys/AnnoSys?providerURL='</code>: root URL for Annosys
***<code>'siteName'=>'my_portal',</code>: the portal name (used in the URL)
***<code>'siteTitle'=>'GGBN Portal',</code>: the default site name/title
***<code>'catalogNumberClickable'=>true,</code>: if the catalogNumber/UnitID can be clickable on the details page

=Personalisation=
If you want to change the icon (favicon), it is located in frontend/web. 
If you want to change the slider images on the home page, they are located in frontend/web/images/slider. Then you will have to modify the view in frontend/view/site/index.php to load the pictures you chose. 
If you want to change the menu icons, they are located in frontend/web/images/icons. 
 
The CSS and JS files are also in the frontend/web folder.

=Adding new search fields or fields to display=
==Adding them in SOLR==
*edit the schema.xml file in SOLR and add the new fields you want to have. If you want to search for them, edit the file for the search core *_search/conf/schema.xml. If wou want to display them in the preview, edit the file for the preview core *_preview/conf/schema.xml. If you need it for the details page, edit the file for the details core *_details/conf/schema.xml. Generally: if you want to search for it (search core), the field has to be indexed, but not mandatory stored. If you want to display it (preview/details), it has to be stored but not indexed. The tripleidstoreid or the occurrenceid *always* has to be indexed.
*restart Tomcat
'''Important ''': always restart Tomcat (service tomcat8 restart) when you modify the schema.xml file.
*edit the Java file corresponding to the core that need to be updated- for example, if you want to index or store the recordbasis, add the following method
<code>
private void addRecordbasis(int datasourceid) throws ClassNotFoundException, SQLException {
System.out.println("RUN RECORDBASIS ");
if(tripleidstoreids.size()>0){
String queryTID =
"SELECT distinct fk_tripleidstoreid as tripleidstoreid, "
+ "IFNULL(recordbasis,'N/A') as recordbasis "
+ " FROM recordbasis "
+ " JOIN occurrence ON recordbasisid = fk_recordbasisid "
+ " WHERE fk_tripleidstoreid in ("+StringUtils.join(tripleidstoreids,", ")+") "
+ " AND fk_datasourceid ="+datasourceid
+ " ORDER BY fk_tripleidstoreid ";
update( queryTID,tripleidstoreids.size());
}
System.gc();
}
</code> and add a call to that method in the '''handleDocuments''' method:
<code>
occurrencesDone.clear();
multivalueForbidden=false;
addRecordbasis(datasourceid );
</code>
*existing documents can not be updated in SOLR in our case (only if everything was "stored"). So first, delete the old data from the core. Depending on how many records there are, you might have to delete the data for a specific repository only solrServer:solrPort/solr/coreName_SEARCH_OR_PREVIEW_OR_DETAILS/update?stream.body=<delete><query>YOUR_LIMITATION_FIELD:*</query></delete>&commit=true This "limitation_field" must be an indexed value in the core. If you don't have a lot of data, you can put <query>*:*</query> as the generation of the new SOLR index will be fast enough (5-10min per Core for 180.000 records).
*per default, it will index this new field for each datasource. If it's still missing at the end for some datasources, or if you only want to do it for specific datasources, edit the '''main''' method <code>for (int i=0;i<166;i++)</code>

==Adding them in the portal==
*Edit the SearchController file (controllers)(additions can be made in the CommonSearchController, deletion only in the specific SearchController if you have several portals running in parallel with the same installation)
**if it's a dropdownlist, add the name of the field in the variable <code>$knownLParams</code>.
**if it's a simple term or a suggestion list, add the name of the field in the variable <code>$knownParams</code>.
*Edit the SearchForm file (models)(same rule as above), edit the variable <code>$_parameters</code> - it can be a text, a select(dropdownlist), a suggest, a radio or a range. Also add the public variable with the same name (ie. <code>public $sampleavailability;</code> and
<code>
'sampleavailability' => [
'type' => 'radio',
'name' => 'Sample availability (loan)',
'config' => [
'size' => 285,
'maxlength' => 150,
'name' => 'sampleavailability'
],
'hideable' => 'true'
]
</code>. It will be automatically displayed in the search form. The '''hideable''' parameter specifies if it is displayed per default or if it is only in the accordion.
Eventually, edit the '''attribteLabels''' function such as <code>'sampleavailability' => 'Sample availability (loan)'</code>
*Edit the listView.php (views) file if it has to be displayed here, or the recordViewTabs.php if it has to be displayed on the details page. Here again, add it to the $suggestFields or $defaultFields or $selectFields or $buttonFields, depending of its type.
*Edit the file CreateFormManager(vendor) only if it's a radio button (look how it's done for cites and sampleavailability).
*Edit the file SOLRQueryManager(vendor) - for example, if it has to look in the SOLR case-unsensitive value, do as for
<code>
case 'fullScientificName' :
return $field . '_nc:' . $value . '';
break;
</code> If it's case sensitive,
<code>
case 'collectioncode' :
return $field . ':"' . $value . '"';
break;
</code>

=Running multiple GPS portals in parallel=

*in common/config/bootstrap: the name of all folders that Yii2 should go through (for imports and so)
*in environnments/index.php: the name of the folders that will be exposed and therefore where Yii should be authorized to write logs and handle the CSS files

=GGBN specific configuration=

The institution has to be filled in bio_datasource (city and institution, for the harvester_factoy like '%Harvester%') - but first the datasource has to be added (metadata update). It will be used for the repository/registry field in the GGBN portal.
The parentInstitution table is used for the contacts.

=Debug=
If something does not work as expected, have a look in the catalina.out file (generated by Tomcat) on the machine where SOLR is running. You will see the last queries sent to SOLR at the end of the file.
The YII file runtime/log/app.log might also help, and is located in the yii folder on the machine running the portal (ie. advanced/runtime....).

Main Page

2016-02-03T08:54:51Z

GabiDroege: /* Adding new search fields or fields to display */

'''Welcome to the Wiki of the GGBN Portal Software'''

Further information about the software will follow soon.

=Presentation=
TBD

=Workflow=
TBD

=Installation=

==You will need==
* Apache (tested with Apache/2.4.7 (Ubuntu), Apache/2.2.22 (Debian), Apache/2.4.10 (Win64)) with libapache2-mod-php5
* PHP5 (tested with 5.5.9, 5.5.30 and 5.5.15). You also might need to install php5-xsl, php5-gd and php5-mysql.
* a SOLR instance (tested with Apache SOLR 4.9.0). Contact us for getting the appropriate SOLR configuration files.
* a MySQL database (tested with MySQL 5.5.46). It will contain data harvested thanks to [http://wiki.bgbm.org/bhit/index.php/Main_Page B-HIT]

Yii (version 2) will be fully contained in the subversion code of the portal software.

==Copy a working version from the svn ==
Create a destination folder, for the rest of this documentation we will choose "my_portal" in your web-folder (ie. /var/www/, or *WINPATH*). 
Get http://ww2.biocase.org/svn/dnabank/Dnabank_Portal/ggbn_portal/trunk/ into "my_portal".

=Configuration=
''TBD''
''yii, Apache, rights, SOLR (SOLR config files also & Java)'' 
First, let say we want to access the portal under the URL http://localhost/ggbn_portal.

*Check that the Apache User has writing/editing rights in the whole "my_portal" folder.
*Edit the file "my_portal/frontend/config/main.php".
**Configure the portal URL: <code>$baseUrl = str_replace ( '/SUBDIR_AFTER_YOUR_WEB_FOLDER/my_portal/frontend/web', '/ggbn_portal', (new Request ())->getBaseUrl () );</code>
**Configure the MySQL database credentials <code>
'db' => [
'class' => '\yii\db\Connection',
'dsn' => 'mysql:host=IP;dbname=DBNAME',
'username' => 'USER',
'password' => 'PASS',
'charset' => 'utf8'
],</code>
*Edit the file "my_portal/frontend/config/params.php".
**e-mails
***Configure the admin e-mail <code>'adminEmail' => 'ADMIN@DOMAIN',</code>. The admin will get error/bug messages.
***Configure the support e-mail(s) <code>'supportEmail' => ['MAIL1','MAIL2'..],</code>.
***Configure the feedback e-mail(s) <code>'feedbackMail' => ['MAIL1','MAIL2'..],</code>. They will get the messages sent with the feedback function.
***Configure the no-reply-mail (who is supposed to sent e-mail(s) to the users) <code>'noreplyMail'=>'MAIL'
**Configure the location of the DarwinCore Archives (harvested with B-HIT) <code>'dwcArchivesPath' => 'PATH_TO_THE_PARENT_FOLDER' ,</code>. Check the access rights! (read is enough)
**Configure the location(root) of your SOLR instance <code>'solrhost'=> 'SOLR_HOST'</code> and the SOLR core names
<code>
$searchname="SEARCH_CORE";
$previewname="PREVIEW_CORE";
$detailsname="DETAILS_CORE";
</code>
**Set the temporary folder <code>'tmpFolder'=>'TEMP_FOLDER'</code>. Check the access rights! (write is needed)
**Configure the options
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'noBack'=>true_OR_false</code>: a special backbone is used
***<code>'shopping'=>true_OR_false</code>: the shopping system is enabled or disabled
***<code>'useLogin'=>true_OR_false</code>: is the login enabled or disabled
***<code>'newsService'=>true_OR_false</code>: uses the GGBN news services - if it has to be used with a different wiki news, edit the SiteController.py
***<code>'displayMore'=>true_OR_false</code>: displays the categories with internal links on the details page
***<code>'displayExternals'=>true_OR_false</code>: displays the categories with external links (GBIF, BOLD, NCBI...) on the details page
***<code>'ABCDdownloable'=>true_OR_false</code>: ABCD files are downloadable or not
***<code>'useAnnosys'=>true_OR_false</code>: link to annosys (annotation system) on the details page
***<code>'annosysAP'=>'https://annosys.bgbm.fu-berlin.de/AnnoSys/AnnoSys?providerURL='</code>: root URL for Annosys
***<code>'siteName'=>'my_portal',</code>: the portal name (used in the URL)
***<code>'siteTitle'=>'GGBN Portal',</code>: the default site name/title
***<code>'catalogNumberClickable'=>true,</code>: if the catalogNumber/UnitID can be clickable on the details page

=Personalisation=
If you want to change the icon (favicon), it is located in frontend/web. 
If you want to change the slider images on the home page, they are located in frontend/web/images/slider. Then you will have to modify the view in frontend/view/site/index.php to load the pictures you chose. 
If you want to change the menu icons, they are located in frontend/web/images/icons. 
 
The CSS and JS files are also in the frontend/web folder.

=Adding new search fields or fields to display=
==Adding them in SOLR==
*edit the schema.xml file in SOLR and add the new fields you want to have. If you want to search for them, edit the file for the search core *_search/conf/schema.xml. If wou want to display them in the preview, edit the file for the preview core *_preview/conf/schema.xml. If you need it for the details page, edit the file for the details core *_details/conf/schema.xml. Generally: if you want to search for it (search core), the field has to be indexed, but not mandatory stored. If you want to display it (preview/details), it has to be stored but not indexed. The tripleidstoreid or the occurrenceid *always* has to be indexed.
*restart Tomcat
'''Important ''': always restart Tomcat (service tomcat8 restart) when you modify the schema.xml file.
*edit the Java file corresponding to the core that need to be updated- for example, if you want to index or store the recordbasis, add the following method
<code>
private void addRecordbasis(int datasourceid) throws ClassNotFoundException, SQLException {
System.out.println("RUN RECORDBASIS ");
if(tripleidstoreids.size()>0){
String queryTID =
"SELECT distinct fk_tripleidstoreid as tripleidstoreid, "
+ "IFNULL(recordbasis,'N/A') as recordbasis "
+ " FROM recordbasis "
+ " JOIN occurrence ON recordbasisid = fk_recordbasisid "
+ " WHERE fk_tripleidstoreid in ("+StringUtils.join(tripleidstoreids,", ")+") "
+ " AND fk_datasourceid ="+datasourceid
+ " ORDER BY fk_tripleidstoreid ";
update( queryTID,tripleidstoreids.size());
}
System.gc();
}
</code> and add a call to that method in the '''handleDocuments''' method:
<code>
occurrencesDone.clear();
multivalueForbidden=false;
addRecordbasis(datasourceid );
</code>
*existing documents can not be updated in SOLR in our case (only if everything was "stored"). So first, delete the old data from the core. Depending on how many records there are, you might have to delete the data for a specific repository only solrServer:solrPort/solr/coreName_SEARCH_OR_PREVIEW_OR_DETAILS/update?stream.body=<delete><query>YOUR_LIMITATION_FIELD:*</query></delete>&commit=true This "limitation_field" must be an indexed value in the core. If you don't have a lot of data, you can put <query>*:*</query> as the generation of the new SOLR index will be fast enough (5-10min per Core for 180.000 records).
*per default, it will index this new field for each datasource. If it's still missing at the end for some datasources, or if you only want to do it for specific datasources, edit the '''main''' method <code>for (int i=0;i<166;i++)</code>

==Adding them in the portal==
*Edit the SearchController file (controllers)(additions can be made in the CommonSearchController, deletion only in the specific SearchController if you have several portals running in parallel with the same installation)
**if it's a dropdownlist, add the name of the field in the variable <code>$knownLParams</code>.
**if it's a simple term or a suggestion list, add the name of the field in the variable <code>$knownParams</code>.
*Edit the SearchForm file (models)(same rule as above), edit the variable <code>$_parameters</code> - it can be a text, a select(dropdownlist), a suggest, a radio or a range. Also add the public variable with the same name (ie. <code>public $sampleavailability;</code> and
<code>
'sampleavailability' => [
'type' => 'radio',
'name' => 'Sample availability (loan)',
'config' => [
'size' => 285,
'maxlength' => 150,
'name' => 'sampleavailability'
],
'hideable' => 'true'
]
</code>. It will be automatically displayed in the search form. The '''hideable''' parameter specifies if it is displayed per default or if it is only in the accordion.
Eventually, edit the '''attribteLabels''' function such as <code>'sampleavailability' => 'Sample availability (loan)'</code>
*Edit the listView.php (views) file if it has to be displayed here, or the recordViewTabs.php if it has to be displayed on the details page. Here again, add it to the $suggestFields or $defaultFields or $selectFields or $buttonFields, depending of its type.
*Edit the file CreateFormManager(vendor) only if it's a radio button (look how it's done for cites and sampleavailability).
*Edit the file SOLRQueryManager(vendor) - for example, if it has to look in the SOLR case-unsensitive value, do as for
<code>
case 'fullScientificName' :
return $field . '_nc:' . $value . '';
break;
</code> If it's case sensitive,
<code>
case 'collectioncode' :
return $field . ':"' . $value . '"';
break;
</code>

=Running multiple GPS portals in parallel=

TBD - short description of YII folder structure (MVC), link to Yii documentation; difference frontend-folder and the rest

=GGBN specific configuration=

The institution has to be filled in bio_datasource (city and institution, for the harvester_factoy like '%Harvester%') - but first the datasource has to be added (metadata update). It will be used for the repository/registry field in the GGBN portal.
The parentInstitution table is used for the contacts.

=Debug=
If something does not work as expected, have a look in the catalina.out file (generated by Tomcat) on the machine where SOLR is running. You will see the last queries sent to SOLR at the end of the file.
The YII file runtime/log/app.log might also help, and is located in the yii folder on the machine running the portal (ie. advanced/runtime....).

Main Page

2015-12-16T15:05:49Z

GabiDroege:

'''Welcome to the Wiki of the GGBN Portal Software'''

Further information about the software will follow soon.