Data processing

Documentation / Enterprise Edition / User Guide / Agent / Data processing

Agent data processing

The MyARM agent implementation measures and processes timing information about the instrumented application. To minimize the interference the MyARM agent implementation uses a threading pipeline as shown in Figure "Agent data processing thread pipeline" to process the measured data.

Figure: Agent data processing thread pipeline

Threads are shown in the light orange (application thread) and blue (MyARM agent threads) boxes marked with the following digits in parenthesis:

  1. Application thread executing application code which is instrumented with calls to the ARM 4.0 interface API.
  2. Serialize thread which will serialize the measured information into a so-called ARM data buffer.
  3. Sink thread which will export the measured data to a destination (database, file or daemon)
  4. Logger thread which will write any log message which is generated by the MyARM agent implementation to a file or logging service (syslog, eventlog).

A typical ARM response time measurement (ARM transaction) is shown in Figure "Agent data processing thread pipeline" with the following letters in parenthesis:

  1. Within the application thread the ARM 4.0 API is called to start a transaction within the ARM agent implementation. The MyARM agent will get a so-called transaction instance from its internal transaction instance memory pool.
  2. The application will continue its work which will be measured by ARM.
  3. After the application has finished its work it will call the ARM 4.0 API indicating that the current ARM transaction has finished by calling the stop transaction function (C) or method (Java). The MyARM agent will record all measured and provided context information within the transaction instance object obtained within (a). Then it will put this transaction instance in the queue of transaction instances of the Serialize thread .
  4. The Serialize thread will be notified that there is a transaction instance in its queue and will process it by serializing the contents in a so-called ARM data buffer:
    • It will get this transaction from the queue
    • Check if the transaction instance fits into the current ARM data buffer. If not the current ARM data buffer is full and is put into the queue of the Sink thread.
    • Check if there is a valid ARM data buffer and if not (ARM data buffer was full or its the first transaction instance) it will get a new ARM data buffer from the ARM data buffer memory pool.
    • Now we have a valid and big enough ARM data buffer and will serialize the contents of the transaction instance into the ARM data buffer.
    • The last step is to release the transaction instance and to put it in the transaction instance memory pool again.
  5. The Sink thread will be notified that there is an ARM data buffer in its queue and will process it by writing the data into the configured datasink:
    • It will get the ARM data buffer from the queue
    • Write the contents of the data buffer to the configured datasink (file, tcp or database)
    • Release the ARM data buffer by putting the ARM data buffer back to its memory pool.
    • Last it will issue a log message containing the number of processed ARM data entries (application and/or transactions). To do this a new logmsg object is obtained from the log message memory pool. Filled with all relevant information and put into the Logger thread queue.
  6. The Logger thread will be notified that there is a log message in its queue:
    • It will get the logmsg object from the queue.
    • Format the message to a readable string.
    • Write the message to the configured logging destination.
    • Release the logmsg object by putting it into the log message memory pool back.