One of the simplest ways to explain QRadar’s architecture is to follow the flow of data through it. This means, what input data is fed into QRadar, and how different components process this data, to produce useful information.
QRadar primarily receives three different types of inputs. These are:
1 – Events:
These are essentially the logs that get generated due to some activity. This can be a log event when firewall blocks certain traffic, or if it allows it. It can also be when a user successfully logs on to a device, or if the user tries a wrong password, a log is generated. The main characteristic of event is that it is logged due to an activity and it gives information about that activity in that instant – like traffic denied, or password was wrong.
2 – Flows
Flows are characterized by records that can span some time period. For example a skype call me be a minute or hours long. There is a start time and end time, while duration between start and end time for this record can be days, hours, minutes, seconds etc.
A flow is unique by 5-tuple, namely:
- Source IP
- Dest IP
- Dest Port
This means a flow can be uniquely identified by looking at these 5 items.
3 – Vulnerability Information
Vulnerability information comes from a QRadar component known as the Vulnerability Manager. Aim of this component is to keep a proactive approach for not letting vulnerabilities be exploited for the devices and machines in organization’s network.
This can visualized as follows:
It receives events/ logs from log sources configured to send logs to QRadar. These events are in raw form, which are normalized by DSM modules before they are sent to event processors. Even though the events get normalized, raw events do not get discarded.
The Event Processor receives raw and normalized events from event collectors for analysis. They also store the data both kinds of events (normalized and raw).
If there is need for more data storage to store events and flows, we can use data nodes.
The magistrate’s job is to correlate data from event processors, for finding what might be going on in the network and generate offenses.
QRadar Event Pipeline:
The components described above contribute by forcing data (events and flows) pass through the following pipeline:
The event collector collects/ receives events from log sources the passes them to license throttling service, which checks if the amount of events coming in to the pipeline are supported by the license. Once the license is checked, the events get sent to the DSM modules for parsing. DMSs are specialized modules which are tailor made to parse logs coming from different devices and different vendors.
The parsed events then go through coalescing filter, where de-duplication of events/ logs happens. After coalescing filter is done “removing” duplicate events, CRE (Custom Rules Engine) Processor takes over. CRE contains the rules which specify when to fire an offense or react in a certain way, when a condition is matched.
For example, we can make a rule in CRE, which generates an offence and notifies QRadar administrator if a new user with administrative privileges is created on active directory.
This pipeline is shown in the flow chart below: