Filter Statistics
Filters are a core foundation of how processing works in NIM, serving an essential component that manages and manipulate data streams. In NIM, filters are used to apply transformations, extract specific data, and perform various operations that are crucial for the efficient functioning of the system. Understanding the duration it takes for a filter to process data is vital for optimizing performance and ensuring smooth data flow. This knowledge allows developers to identify potential bottlenecks, balance workloads, and enhance the overall responsiveness of the system. By carefully monitoring and adjusting filter processing times, NIM can maintain high efficiency and reliability, crucial for handling complex data processing tasks.
Every time a filter executes, a log message is generated. All times listed on the log entry in milliseconds (ms). Below is an example
2022-08-08T12:25:44.093Z verbose: Filter stats fManagerEmployees (ms): total:5, query:4, resolve:0, docs:1, sort:0, ordering:0, exclude:0, lookup:0, append:0 #21/0/1 (initialization) { "fields": [ { "fieldName": "ID", "fieldType": "STRING", "readAccessType": "ADMINS_AND_SELF", "multiValued": false, "displayName": "ID" }, { "fieldName": "Type", "fieldType": "STRING", "readAccessType": "ADMINS_AND_SELF", "multiValued": false, "displayName": "Type" }, { "fieldName": "DeleteDate", "fieldType": "STRING", "readAccessType": "ADMINS_AND_SELF", "multiValued": false, "displayName": "DeleteDate" } ], "schemaName": "Tools4ever", "displayName": "Tools4ever"}
total: The total time to execute the filter
query: Time to initialize parameters and variables and calculate the raw query result
resolve: Time to calculate the result document references based on the raw query result and the dataset
docs: Time to compose the resulting documents
sort: Time to sort the resulting documents
ordering: Time to determine the included and excluded documents
exclude: Time to process excluded documents based on specific column
lookup: Time to process the lookup specification, if any
append: Time to append results of other specified filter.
#21/0/1:
Number of included documents (21)
Number of documents removed by lookup operation (0)
Number of documents excluded (1)