Gis Integration

The following wiki details the gis_integration application which handles api calls for transferring kmz files from a specified endpoint to the NocPortal.

Extenet wanted to automate the retrieval and storage of kmz files in the NocPortal Previously this was a manual task :

• Extenet would export any updated KMZ's from ArcGis
• Ops Request
• Manual load of KMZ's into portal via Manage overlays.

• Springboot 2.1.4
• Gradle
• Apache camel 2.23.2 (starter, http, jackson)
• Okhttp 3.12.0
• mysql-connector 5.1.47 (setup for db but not used currently)

Cron job Scheduler configurable via /var/spring-boot/gis_integration/application.properties

scheduler.cronTrigger.expression=0 0 3 * * *

Mail Setup for Success / Failure Notifications configurable via /var/spring-boot/gis_integration/application.properties

(Differ per environment - templated in env-configuration)

spring.mail.enabled=true
spring.mail.port=587
spring.mail.host=smtp.gmail.com
spring.mail.username=rightofway@errigal.com
spring.mail.password=#Rightofway321!#

gis.mail.from=kevin.fan@errigal.com
gis.mail.replyTo=support@errigal.com
gis.mail.recipients=kevin.fan@errigal.com

NocPortal API endpoints configurable via /var/spring-boot/gis_integration/application.properties

(Differ per environment - templated in env-configuration)

nocportal.integration.states.url=http://localhost:8080/NocPortal/api/v1/geographicLocation
nocportal.integration.overlay.url=http://localhost:8080/NocPortal/api/v1/mapOverLay
nocportal.integration.upload.timeout=30000
nocportal.api.userName=userapiadmin
nocportal.api.password=notsame1

KMZ File Storage configurable via /var/spring-boot/gis_integration/application.properties

file.directory.location=kmzfiles/
file.directory.retention.period=-7

ArcGis API endpoints configurable via /var/spring-boot/gis_integration/application.properties

(Differ per environment - templated in env-configuration)

remote.timeout=60000
remote.tokenUrl=https://gis.extenetsystems.com/arcgis/tokens/generateToken?httpClient.soTimeout=
remote.username=mzubal_errigal
remote.password=mzubal_errigal123
remote.client=remoteip
remote.expiration=60
remote.request.format=json
remote.request.referer=https://www.arcgis.com
remote.queryUrl=https://gis.extenetsystems.com/arcgis/rest/services/FeaturedContent/Errigal_CL_TEST/MapServer/6/query?where=ProjectState=''&geometryType=esriGeometryEnvelope&spatialRel=esriSpatialRelIntersects&returnGeometry=true&returnTrueCurves=false&returnIdsOnly=false&returnCountOnly=false&returnZ=false&returnM=false&returnDistinctValues=false&returnExtentOnly=false&f=kmz
remote.whereQuery=where=ProjectState=''       

  * GET /api/v1/geographicLocation
  * POST /api/v1/mapOverLay

  * /var/spring-boot/gis_integration/gis_integration.log

Monitoring is configured for the following servers:

1. sfqaapps1
2. qaextapps1
3. extapps1

Actuator setup tutorial: https://www.callicoder.com/spring-boot-actuator/

Metrics setup tutorial: https://www.callicoder.com/spring-boot-actuator-metrics-monitoring-dashboard-prometheus-grafana/

Added dependencies to build.gradle:

 compile("org.springframework.boot:spring-boot-starter-actuator")

// Micrometer + Prometheus exporter
compile(group: 'io.micrometer', name: 'micrometer-core')
compile(group: 'io.micrometer', name: 'micrometer-registry-prometheus') 

Added config to application.properties:

# Metrics related configurations
management.endpoint.metrics.enabled=true
management.endpoints.web.exposure.include=*
management.endpoint.prometheus.enabled=true
management.metrics.export.prometheus.enabled=true
management.metrics.tags.application=gis_integration

prometheus–monitoring-config:

• gis_integration-sd.yml
• prometheus.yml (GIS INTEGRATION entry)

1. Check failure email for exception - This will be emailed to operations
2. Check endpoints are reachable (ArcGis and NocPortal)
3. Restart Application if necessary or wait for the next cron schedule to run.
4. ensure gis_integration app is running on apps1 handler
5. Check developer console for errors with KMZ file when loading

• Sunday Aiyash saiyash@extenetsystems.com (Manages Data Integrity (GIS) team)
• Vishal Shah vshah@extenetsystems.com
• Ahmad Saleh asaleh@extenetsystems.com

* Coy Potts cpotts@extenetsystems.com ( No longer part of GIS team)

If for some reason you need to login to the gis application and manually download a state kmz file you can do so as follows:

Login to the following url using the credentials in application.properties (remote.username, remote.password): https://gis.extenetsystems.com/arcgis/rest/services/FeaturedContent/ExteNetFiberAssets_Errigal/MapServer/6/Query

Use the following field population in the form to export the kmz (things to note: ProjectState = 'StateName' & Format = kmz)