Five foundations for tackling cybersecurity challenges

  1. Scale up data integration and management
  2. Automate collection and ingestion of data at data Lake
  3. Capability of fast data retrieval and search.

Cyber analytics platforms must be able to spot threats across a wide variety of data sources – both internal and external to the enterprise – be it real-time streaming data or batch data (i.e., structured, semi-structured and unstructured data formats). Most important, these platforms must work with data that is beyond the traditional purview of security operations teams, such as email content, social media feeds, user metadata from a human resources database, web server and system logs of user activity, as well as critical auditing databases managed by IT teams.

Furthermore, many systems are inherently limited in terms of the amount of data they can store, ranging from a week to a month in some cases. Imagine the scenarios when a vulnerability is discovered 100 days after the fact, and the source data is purged every 90 days. Adding to this is the data associated with mobile devices, IoT devices and cloud-based services, totalling thousands of gigabytes every second. Therefore, it is critical to anticipate potential future use cases and source the data in real-time and scale and store it in a manageable way. This will enable security teams to establish historical baselines to perform investigative experiments and historical analysis.

  • Utilize an integrated advanced analytics-driven platform.
  • De-fragment and reconcile siloed data for rapid insight generation.
  • Power analysis with ML and other advanced forms of AI.
  • Use AI and automation to close skills gaps.

Fragmented data results in fragmented investigation and forensic analysis. Cybersecurity requires an integrated and intelligent analytics-based platform that can automate scanning at the scale and speed required to process increasingly data patterns.

The cyber analytics platform must be able to crunch massive volumes of disparate data and derive meaningful insights, convert data into intelligent information and detect advanced threats using data science, deep learning, edge analytics and AI. By applying advanced analytics technologies user behaviour and risk quotients; proactive identification of vulnerability gaps; and use of social media data to track potential local security incidents.

Automated orchestration is critically needed in the case of zero-day exploits.

  • Seek real-time data enrichment.
  • Add structure and context with metadata such as geo-IP lookups.
  • Add streaming analytics for real-time alerts.

Simply collecting large volumes of data without preparing it for analysis can result in a data burst and huge issues of storage. The cyber analytics platform must be able to correlate patterns among disparate sources of data. For example, legacy systems often send data with timestamps but no indication of time zone. Without that information, SOC analysts cannot be certain of where and when an event was triggered to correlate it with events from other sources with different time zones.

The cyber analytics platform must enrich enterprise event data by tagging critical metadata such as unique host names, geolocation, time zone, etc. as soon as it is ingested.

  • Apply intelligent visualization.
  • Create a customizable command centre view for comprehensive security dashboard.
  • Facilitate integrations for business intelligence tools.

With traditional SOC dashboards and vendor-specific information security tools, incident analysis involves switching between several consoles and user interfaces and performing manual checks and static analysis on data to determine root cause while maintaining sequence of investigations. Each step needs to be repeated for each triggered alert. This manual method of analysis and reporting is highly time-consuming, prone to human error.

It takes dedicated personnel to maintain and monitor such siloed dashboards and perform analysis. An SOC analyst specializing in an individual vendor-based information security tool may not be able to correlate the events from a parallel source of information from a different tool.

The cyber analytics platform must provide SOC analysts with a single view of current IT risk and health scores, as well as a digital map connecting the dots between thousands of people, machines and devices and their interactions. It must also provide the flexibility to create purpose-built dashboards that present intelligent information from correlated data and insights derived from advanced analytics such as real-time behaviour profiling.

  • Expand the security analysis surface via the cloud.
  • Extend the boundaries of data gathering never exclude or ignore cloud.
  • Deploy cloud-native security tools.

As enterprise perimeters expand to the cloud via IoT, IT organizations need solid cloud security protocols and a holistic view of the user and system activity patterns across on-premises and cloud environments. With immature security auditing and governance capabilities in the cloud, threat vectors for data leakage and exfiltration can increase substantially.

Consider a scenario in which an employee uploads data and files from the office laptop to cloud storage that is open to public access.

With cloud-native security products, organizations can better identify cloud assets, which is critical when dealing with vague cloud-generated private IPs across multiple cloud accounts.

Bringing it all together

Below figure depicts an end-state high-level reference architecture of a conceptual next-gen cyber analytics platform.

Such a platform can now be conceived and built easily by integrating industry-standard advanced analytics tools and big data technology.

Organisations should deliver the capability and try to avoid the intervention of dedicated personnel’s for preparing the customised dashboards for the enterprises need. They should try and deliver the KPI/Matrices in the dashboard which should update automatically and should be pivoted to the last level of reporting. This way, Organisation can resolve the talent issues in the cyber security space dedicate resource in the required field which is protection mitigation and if need in the investigation area.

A next-gen cyber analytics platform- Enterprise Wide

Organisation can either built SOC/SIEM combination by creating the bespoke data-lake, Analytics tool, log servers, SOAR, UEBA, threat feeds and intelligence and best of breed selection of SIEM tools like Qradar, Splunk, Fireeye- Helix, LogRhythm, Gurukul etc. other option is to evaluate the capability of the above mentioned features in the proposed SIEM/SOC products mentioned earlier. Organisation need to integrate multiple technologies, special attention to the dashboard requirement of a top management e.g. CXO’s and Board and continuously find out the effectiveness of the SIEM and SOC of the organisation in view of the latest attacks.

Organizations can leverage available tools and validate best-of-breed technologies to quickly deliver on the cyber analytics platform vision while also addressing business priorities. Competitive options are also available from popular public cloud vendors such as IBM QRadar on cloud and use capability of AI/ML and data analytics of multiple public cloud offering of AWS SageMaker, Azure Analysis Services and Google Cloud ML.

*NIST cybersecurity framework: https://www.nist.gov/cyberframework.