Differences

This shows you the differences between two versions of the page.

--- onboarding:elastic-replicator:start [2022/10/04 14:20] – 10.91.110.100
+++ onboarding:elastic-replicator:start [2024/08/29 11:28] (current) – 10.91.129.119
@@ Line 34: / Line 34: @@
 ===== Scheduled Imports =====
+The bulk import is configured with quartz to run on a schedule (see BulkRouter.java for logic details). This refreshes the data by re-running the queries for all datatypes and updating the existing elastic search indexes. The timing of the cron is configured in the application properties as follows:
+<code>
+bulk.import.quartz.schedule=0 0 1 1/1 * ? *
+</code>
+{{ :onboarding:elastic-replicator:screenshot_2022-10-04_at_14.22.22.png?600 |}}
 ===== On Demand Imports =====
-The date can be imported either via individual or bulk mechanisms:
+The data can be imported either via individual or bulk mechanisms as follows:
 **Individual:**
-  - GET http://10.91.140.218:8100/bulk?routes=trap
+  - GET http://10.91.140.218:8100/elastic-replicator/bulk?routes=trap
-  - GET http://10.91.140.218:8100/bulk?routes=networkElement
+  - GET http://10.91.140.218:8100/elastic-replicator/bulk?routes=networkElement
-  - GET http://10.91.140.218:8100/bulk?routes=ticket
+  - GET http://10.91.140.218:8100/elastic-replicator/bulk?routes=ticket
-  - GET http://10.91.140.218:8100/bulk?routes=alarm
+  - GET http://10.91.140.218:8100/elastic-replicator/bulk?routes=activeAlarm
-  - GET http://10.91.140.218:8100/bulk?routes=uptime
+  - GET http://10.91.140.218:8100/elastic-replicator/bulk?routes=uptime
+**Limited individual**
+  - GET "http://10.91.140.218:8100/bulk?routes=uptime&start=1"
+to limit to 1 month back.
 **Bulk:**
@@ Line 63: / Line 75: @@
-===== SSE =====
+===== SSE(Server Sent Events Process) =====
+The SSE subscription drives real time update receipt in the angular ems-nova front end application.
+Server-Sent Events (SSE) is a server push technology enabling a client to receive automatic updates from a server via an HTTP connection, and describes how servers can initiate data transmission towards clients once an initial client connection has been established.
+In this case nova subscribes to the idms-proxy sse endpoint and listens for updates:
+<code>
+/idms-proxy/api/v0/replicator
+</code>
+The SSE touchpoints for  processing are as follows(see also diagram below):
+  - Gorm listener events for ticket, alarm, element inserts or updates are pushed from Ticketer and SnmpManager onto rabbitmq.
+  - Elastic Replicator consumes messages from these queues, processes them using the sql queries and updates or inserts new records into elastic search. It also pushes the events onto a queue which is consumed by idms-proxy.
+  - idms-proxy consumes a queue which receives event messages from elastic-replicator.
+  - Nova front end is subscribed to proxies sse endpoint /replicator and receives messages for real time page updates.
+{{ :onboarding:elastic-replicator:eventnotificationprocess.drawio.png?600 |}}
   * Analysis document: https://docs.google.com/document/d/1nUBZtjKfTnopua9quoIhOpeugYVlCYHB8udhRcXwbS4/edit#heading=h.e1lpa8mdes4i
   * Requirements document: https://docs.google.com/document/d/14mxFTS-B7BaPDNNERJVtzFEZlsCRDIQTFLbgOLaVQxo/edit#heading=h.euwbsr2vf8os
 ===== Development Configuration =====
-Project can be pointed at local or server deployed version of all api endpoints listed above.
+Project can be pointed at local or server deployed version of all /bulk api endpoints listed above in the import section.
 See src/main/resources/application.properties for environment configuration for local dev.

Sidebar

Internal Errigal Collaboration Wiki

Differences

Sidebar

Internal Errigal Collaboration Wiki

User Tools

Site Tools

Differences

Page Tools