Differences

This shows you the differences between two versions of the page.

--- development:applications:gis_integration [2019/05/22 16:52] – adowling
+++ development:applications:gis_integration [2021/06/25 10:09] (current) – external edit 127.0.0.1
@@ Line 1: / Line 1: @@
+ --- //[[anna.dowling@errigal.com|Anna Dowling]] 2019/05/17 05:29//
+====== Gis Integration ======
+The following wiki details the gis_integration application which handles api calls for transferring kmz files from a specified endpoint to the NocPortal.
+===== Use Case =====
+Extenet wanted to automate the retrieval and storage of kmz files in the NocPortal
+Previously this was a manual task :
+  * Extenet would export any updated KMZ's from ArcGis
+  * Ops Request
+  * Manual load of KMZ's into portal via Manage overlays.
+===== Architecture Diagram =====
+{{ :development:applications:gis_integration_app_1_.png |}}
+===== Technology Stack =====
+  * Springboot 2.1.4
+  * Gradle
+  * Apache camel 2.23.2 (starter, http, jackson)
+  * Okhttp 3.12.0
+  * mysql-connector 5.1.47 (setup for db but not used currently)
+===== Key Configuration Points =====
+**Cron job Scheduler configurable via /var/spring-boot/gis_integration/application.properties**
+<code>
+scheduler.cronTrigger.expression=0 0 3 * * *
+</code>
+**Mail Setup for Success / Failure Notifications configurable via /var/spring-boot/gis_integration/application.properties**
+(Differ per environment - templated in env-configuration)
+<code>
+spring.mail.enabled=true
+spring.mail.port=587
+spring.mail.host=smtp.gmail.com
+spring.mail.username=rightofway@errigal.com
+spring.mail.password=#Rightofway321!#
+gis.mail.from=kevin.fan@errigal.com
+gis.mail.replyTo=support@errigal.com
+gis.mail.recipients=kevin.fan@errigal.com
+</code>
+**NocPortal API endpoints configurable via /var/spring-boot/gis_integration/application.properties**
+(Differ per environment - templated in env-configuration)
+<code>
+nocportal.integration.states.url=http://localhost:8080/NocPortal/api/v1/geographicLocation
+nocportal.integration.overlay.url=http://localhost:8080/NocPortal/api/v1/mapOverLay
+nocportal.integration.upload.timeout=30000
+nocportal.api.userName=userapiadmin
+nocportal.api.password=notsame1
+</code>
+**KMZ File Storage configurable via /var/spring-boot/gis_integration/application.properties**
+<code>
+file.directory.location=kmzfiles/
+file.directory.retention.period=-7
+</code>
+**ArcGis API endpoints configurable via /var/spring-boot/gis_integration/application.properties**
+(Differ per environment - templated in env-configuration)
+<code>
+remote.timeout=60000
+remote.tokenUrl=https://gis.extenetsystems.com/arcgis/tokens/generateToken?httpClient.soTimeout=
+remote.username=mzubal_errigal
+remote.password=mzubal_errigal123
+remote.client=remoteip
+remote.expiration=60
+remote.request.format=json
+remote.request.referer=https://www.arcgis.com
+remote.queryUrl=https://gis.extenetsystems.com/arcgis/rest/services/FeaturedContent/Errigal_CL_TEST/MapServer/6/query?where=ProjectState=''&geometryType=esriGeometryEnvelope&spatialRel=esriSpatialRelIntersects&returnGeometry=true&returnTrueCurves=false&returnIdsOnly=false&returnCountOnly=false&returnZ=false&returnM=false&returnDistinctValues=false&returnExtentOnly=false&f=kmz
+remote.whereQuery=where=ProjectState=''
+</code>
+==== NocPortal API ====
+<code>
+  * GET /api/v1/geographicLocation
+  * POST /api/v1/mapOverLay
+</code>
+==== Log Location ====
+<code>
+  * /var/spring-boot/gis_integration/gis_integration.log
+</code>
+====Prometheus setup ====
+Monitoring is configured for the following servers:
+  - sfqaapps1
+  - qaextapps1
+  - extapps1
+==== Springboot 2 Prometheus Setup ====
+Actuator setup tutorial:
+https://www.callicoder.com/spring-boot-actuator/
+Metrics setup tutorial:
+https://www.callicoder.com/spring-boot-actuator-metrics-monitoring-dashboard-prometheus-grafana/
+Added dependencies to build.gradle:
+<code>
+ compile("org.springframework.boot:spring-boot-starter-actuator")
+// Micrometer + Prometheus exporter
+compile(group: 'io.micrometer', name: 'micrometer-core')
+compile(group: 'io.micrometer', name: 'micrometer-registry-prometheus')
+</code>
+Added config to application.properties:
+<code>
+# Metrics related configurations
+management.endpoint.metrics.enabled=true
+management.endpoints.web.exposure.include=*
+management.endpoint.prometheus.enabled=true
+management.metrics.export.prometheus.enabled=true
+management.metrics.tags.application=gis_integration
+</code>
+prometheus--monitoring-config:
+  * gis_integration-sd.yml
+  * prometheus.yml (GIS INTEGRATION entry)
+==== Troubleshooting ====
+  - Check failure email for exception - This will be emailed to operations
+  - Check endpoints are reachable (ArcGis and NocPortal)
+  - Restart Application if necessary or wait for the next cron schedule to run.
+  - ensure gis_integration app is running on apps1 handler
+  - Check developer console for errors with KMZ file when loading
+==== EXT GIS Contacts ====
+  * Sunday Aiyash <saiyash@extenetsystems.com> (Manages Data Integrity (GIS) team)
+  * Vishal Shah <vshah@extenetsystems.com>
+  * Ahmad Saleh <asaleh@extenetsystems.com>
+<del>  * Coy Potts <cpotts@extenetsystems.com></del> ( No longer part of GIS team)
+==== GIS Endpoint Manual KMZ Download ====
+If for some reason you need to login to the gis application and manually download a state kmz file you can do so as follows:
+Login to the following url using the credentials in application.properties (remote.username, remote.password): https://gis.extenetsystems.com/arcgis/rest/services/FeaturedContent/ExteNetFiberAssets_Errigal/MapServer/6/Query
+Use the following field population in the form to export the kmz (things to note: ProjectState = 'StateName' & Format = kmz)
+{{ :development:applications:screencapture-gis-extenetsystems-arcgis-rest-services-featuredcontent-extenetfiberassets-errigal-mapserver-6-query-2019-05-22-16_56_13.png |}}

Sidebar

Internal Errigal Collaboration Wiki

Differences

Sidebar

Internal Errigal Collaboration Wiki

User Tools

Site Tools

Differences

Page Tools