Appearance
Receiving data through push ingress
Push ingress lets external systems and users submit data to NOOSAdmin endpoints for storage in tables. It suits situations in which the source can initiate a submission, such as sending a measurement when it becomes available or delivering a collection of records together.
An endpoint provides a configured entry point for that information. It can require authentication, preprocess submitted data and validate inputs before storage. Ingress can also fire a system-wide trigger to support further operations.
Choosing push or pull ingress
The main distinction is which side initiates data transfer. With push ingress, the external source submits information. With pull ingress, NOOSAdmin retrieves it.
| Consideration | Push ingress | Pull ingress |
|---|---|---|
| Who initiates transfer? | An external system or user. | NOOSAdmin. |
| How does data arrive? | Submission to an ingress endpoint. | Retrieval from an external source. |
| Typical situation | A source can send records when available. | A source exposes an HTTP response or downloadable file. |
| Collection timing | The source decides when to submit. | Retrieval can start on a schedule or system-wide trigger. |
| Documented processing options | Validation and embedded TypeScript preprocessing before storage. | Scripted retrieval, authenticated calls, retries and throttling. |
An application can use both approaches. For example, one equipment system might submit readings directly while another publishes periodic CSV exports. Each can participate in the application's collection process through the appropriate ingress mechanism.
Endpoints and destination tables
Push ingress connects submitted information with the tables in which it will be stored. The destination model should describe the information the application needs, including the fields and relationships that give each record meaning.
Before arranging a submission, the integration team needs to establish what the source sends and how that information corresponds to the destination data. In the equipment example, this includes how a reading identifies the relevant equipment and expresses its measurement value.
The endpoint address, submission format and authentication details are specific to the configured integration.
Individual and batched submissions
An ingress endpoint can receive data for storage and can support batched storage of multiple records. Individual submissions are useful when information becomes available one item at a time. A batch is useful when the source already holds a collection ready to deliver together.
Batch support describes the ability to submit multiple records. The integration's handling of invalid records, partial storage and repeated submissions needs to be established separately; batch support alone does not specify those behaviours.
Authentication, preprocessing and validation
These capabilities address different parts of receiving data:
| Capability | Purpose |
|---|---|
| Authentication | Require the submitting party to authenticate to the endpoint. |
| Preprocessing | Use an embedded TypeScript script to prepare incoming information before it is stored. |
| Validation | Check incoming information before storing it in the database. |
For example, a preprocessing script could convert a source-specific representation into values suitable for the destination table. This is an illustrative use of scripting; the actual transformation depends on the source and application model.
Validation takes place before storage. Both preprocessing and validation participate in preparing acceptable data for the database, but their exact relative execution order is not specified here.
Storage and triggers
Once data is stored, tables make it available to the platform's exploration and processing capabilities. When ingress takes place, it can fire a system-wide trigger. A configured operation can use a trigger to initiate work, such as processing associated with new data.
The trigger relationships are part of the application configuration. Receiving information does not, by itself, define which processors or runners should execute or the order of subsequent work.
Example: submitting equipment readings
In an illustrative monitoring application, an external collection service sends readings to an authenticated ingress endpoint. The application uses an Equipment table and a Measurements table, with measurements referring to the equipment they describe.
The team configures any required preprocessing to prepare the submitted values and validation to check the inputs before storage. The service may send readings individually or submit a collection through an endpoint supporting batches.
Users can then inspect the stored measurements in an explorer. If the application needs processing in response to new records, that behaviour can be configured through the relevant trigger mechanism. The resulting calculations and their presentation are separate parts of the application workflow.