--- //[[anna.dowling@errigal.com|Anna Dowling]] 2019/05/17 05:29//

====== Gis Integration ======

The following wiki details the gis_integration application which handles api calls for transferring kmz files from a specified endpoint to the NocPortal. 

===== Use Case =====
Extenet wanted to automate the retrieval and storage of kmz files in the NocPortal
Previously this was a manual task :
  * Extenet would export any updated KMZ's from ArcGis
  * Ops Request
  * Manual load of KMZ's into portal via Manage overlays.

===== Architecture Diagram =====

{{ :development:applications:gis_integration_app_1_.png |}}


===== Technology Stack =====
  * Springboot 2.1.4
  * Gradle
  * Apache camel 2.23.2 (starter, http, jackson)
  * Okhttp 3.12.0
  * mysql-connector 5.1.47 (setup for db but not used currently)

===== Key Configuration Points =====

**Cron job Scheduler configurable via /var/spring-boot/gis_integration/application.properties**

<code>
scheduler.cronTrigger.expression=0 0 3 * * *
</code>


**Mail Setup for Success / Failure Notifications configurable via /var/spring-boot/gis_integration/application.properties**

(Differ per environment - templated in env-configuration)

<code>
spring.mail.enabled=true
spring.mail.port=587
spring.mail.host=smtp.gmail.com
spring.mail.username=rightofway@errigal.com
spring.mail.password=#Rightofway321!#

gis.mail.from=kevin.fan@errigal.com
gis.mail.replyTo=support@errigal.com
gis.mail.recipients=kevin.fan@errigal.com
</code>

**NocPortal API endpoints configurable via /var/spring-boot/gis_integration/application.properties**

(Differ per environment - templated in env-configuration)
<code>
nocportal.integration.states.url=http://localhost:8080/NocPortal/api/v1/geographicLocation
nocportal.integration.overlay.url=http://localhost:8080/NocPortal/api/v1/mapOverLay
nocportal.integration.upload.timeout=30000
nocportal.api.userName=userapiadmin
nocportal.api.password=notsame1
</code>

**KMZ File Storage configurable via /var/spring-boot/gis_integration/application.properties**

<code>
file.directory.location=kmzfiles/
file.directory.retention.period=-7
</code>

**ArcGis API endpoints configurable via /var/spring-boot/gis_integration/application.properties**

(Differ per environment - templated in env-configuration)
<code>
remote.timeout=60000
remote.tokenUrl=https://gis.extenetsystems.com/arcgis/tokens/generateToken?httpClient.soTimeout=
remote.username=mzubal_errigal
remote.password=mzubal_errigal123
remote.client=remoteip
remote.expiration=60
remote.request.format=json
remote.request.referer=https://www.arcgis.com
remote.queryUrl=https://gis.extenetsystems.com/arcgis/rest/services/FeaturedContent/Errigal_CL_TEST/MapServer/6/query?where=ProjectState=''&geometryType=esriGeometryEnvelope&spatialRel=esriSpatialRelIntersects&returnGeometry=true&returnTrueCurves=false&returnIdsOnly=false&returnCountOnly=false&returnZ=false&returnM=false&returnDistinctValues=false&returnExtentOnly=false&f=kmz
remote.whereQuery=where=ProjectState=''       
</code>

==== NocPortal API ====
<code>
  * GET /api/v1/geographicLocation
  * POST /api/v1/mapOverLay
</code>

==== Log Location ====
<code>
  * /var/spring-boot/gis_integration/gis_integration.log
</code>

====Prometheus setup ====
Monitoring is configured for the following servers: 
  - sfqaapps1 
  - qaextapps1
  - extapps1

==== Springboot 2 Prometheus Setup ====
Actuator setup tutorial:
https://www.callicoder.com/spring-boot-actuator/

Metrics setup tutorial:
https://www.callicoder.com/spring-boot-actuator-metrics-monitoring-dashboard-prometheus-grafana/

Added dependencies to build.gradle:
<code>
 compile("org.springframework.boot:spring-boot-starter-actuator")

// Micrometer + Prometheus exporter
compile(group: 'io.micrometer', name: 'micrometer-core')
compile(group: 'io.micrometer', name: 'micrometer-registry-prometheus') 
</code>

Added config to application.properties:
<code>
# Metrics related configurations
management.endpoint.metrics.enabled=true
management.endpoints.web.exposure.include=*
management.endpoint.prometheus.enabled=true
management.metrics.export.prometheus.enabled=true
management.metrics.tags.application=gis_integration
</code>

prometheus--monitoring-config:
  * gis_integration-sd.yml
  * prometheus.yml (GIS INTEGRATION entry)



==== Troubleshooting ====
  - Check failure email for exception - This will be emailed to operations
  - Check endpoints are reachable (ArcGis and NocPortal)
  - Restart Application if necessary or wait for the next cron schedule to run.
  - ensure gis_integration app is running on apps1 handler
  - Check developer console for errors with KMZ file when loading

==== EXT GIS Contacts ====

  * Sunday Aiyash <saiyash@extenetsystems.com> (Manages Data Integrity (GIS) team)
  * Vishal Shah <vshah@extenetsystems.com>
  * Ahmad Saleh <asaleh@extenetsystems.com>
 
<del>  * Coy Potts <cpotts@extenetsystems.com></del> ( No longer part of GIS team)


==== GIS Endpoint Manual KMZ Download ====
If for some reason you need to login to the gis application and manually download a state kmz file you can do so as follows:

Login to the following url using the credentials in application.properties (remote.username, remote.password): https://gis.extenetsystems.com/arcgis/rest/services/FeaturedContent/ExteNetFiberAssets_Errigal/MapServer/6/Query

Use the following field population in the form to export the kmz (things to note: ProjectState = 'StateName' & Format = kmz)

{{ :development:applications:screencapture-gis-extenetsystems-arcgis-rest-services-featuredcontent-extenetfiberassets-errigal-mapserver-6-query-2019-05-22-16_56_13.png |}}