Professional Documents
Culture Documents
Lab9 Using Mine Meld For IoC Feed Aggregation
Lab9 Using Mine Meld For IoC Feed Aggregation
For access to live Palo Alto Networks lab boxes, go to: https://www.paloaltonetworks.com/services/education/cybersecurity-skills-practice-lab
Overview
In order to prevent successful cyberattacks, many organizations collect indicators of
compromise (IOCs) from various threat intelligence providers with the intent of creating new
controls for their security devices. Unfortunately, legacy approaches to aggregation and
enforcement are highly manual in nature, often creating complex workflows and extending the
time needed to identify and validate which IOCs should be blocked.
Objective
The purpose of this lab is to familiarize you with Palo Alto’s MineMeld product and to
demonstrate how to aggregate multiple threat feeds to utilize the threat intelligence on a Palo
Alto firewall. By the end of the lab we will have a MineMeld instance configure, know how to
configure threat feeds, and populate our Palo Alto firewall with IoC and SaaS service
information.
Target Device
Palo Alto Firewall(s) with PANOS 7.1.x or greater
MineMeld VM
Getting Started
Though there are multiple ways to install MineMeld for the purpose of this lab we will
be using the OVA image that has MineMeld preconfigured.
1. To set up a lab installation, Download MineMeld OVA from the following link:
https://s3-eu-west-1.amazonaws.com/minemeld-dist/0_9/minemeld-vm-0.9.10-1build1.ova
The OVA is an original Ubuntu 14.04 image with a preconfigured MineMeld instance.
1. In VMWare - Select File > open.. and brose to the location where the OVA was
downloaded.
2. Name the VM and select the path where you wish to install the instance.
Sun Mgt Bonus Lab 9: Using Mine Meld for IoC Feed Aggregation 3
For access to live Palo Alto Networks lab boxes, go to: https://www.paloaltonetworks.com/services/education/cybersecurity-skills-practice-lab
NOTE. If VMWare throws an error about the OVF format just press ‘Retry’
The VM will be provisioned with minimal resources. For the purpose of this lab the default
setting is adequate.
1. Start the VM, Once the system is booted the Default credentials for shell access are:
Username: ubuntu
Password: rsplizardspock
2. Upon boot the MineMeld image will check for the latest version (as of this writing 9.44
is current release) if there is outbound connectivity. It is good practice to verify that you
are running the latest instance. To update MineMeld once booted issue:
>sudo minemeld-auto-update
3. Obtain the IP of the instance and are ready to log into MineMeld. Issuing ‘ifconfig’ at the
cli and taking note of the IP.
Username: admin
Password: minemeld
MineMeld Concepts
It is important to familiarize yourself with some of the basic concepts that MineMeld
uses. The core concepts to understand are Nodes, Miners, Process, and Outputs.
Miners and are responsible for periodically retrieving indicators from different feeds and
pushing them downstream to the connected nodes using update messages. Miners are also
responsible for aging out indicators: when indicators disappear from the original feed or when
an indicator is considered dead, the corresponding Miner instructs the downstream nodes of
removing the indicator via a withdraw message.
From the above graph the central red node is a Processor node. In this specific
configuration, the processor node is an IPv4 aggregator node and aggregates IPv4 indicators
received from the 5 Miners and sends downstream the aggregated indicators.
The 3 yellow nodes on the right are Output nodes. These nodes receive indicators from
the processor nodes and transform them into a format that could be directly consumed by Palo
Alto firewalls. In the default config the 3 output nodes translate the indicators received from
Sun Mgt Bonus Lab 9: Using Mine Meld for IoC Feed Aggregation 7
For access to live Palo Alto Networks lab boxes, go to: https://www.paloaltonetworks.com/services/education/cybersecurity-skills-practice-lab
the aggregator node into a format that can be consumed using the PAN-OS External Dynamic
List (EDL) feature. All 3 output nodes in this graph receive the same set of indicators from the
aggregator node, but each of them stores a different subset of these indicators based on the
configured input filters. inboundfeedhc accepts only indicators with confidence level > 75,
inboundfeedmc only indicators with confidence level < 75 and > 50, inboundfeedlc indicators
with confidence level < 50. These subsets of indicators are stored into 3 different EDLs that can
be used in different ways inside the PAN-OS configuration.
Note: Verify you are in ‘expert mode’. The eye in the bottom lower left corner should have a strike
through it.
3. Add ‘zeustracker.badips’ which has a confidence level 100 so will be aggregated into the
high confidence output (inboundfeedhc)
• Name: zeustracker_badips
• Prototype: zeustracker.badips
• Inputs: inboundaggregator
7. The inboundaggregator will now populate the output node which will filter the IoC
based on confidence.
In order to use the IoC data on the firewall we need to add EDLs objects to the firewall. In
MineMeld:
1. Click Config
2. Browse to inboundfeedhc and inboundfeedmc
3. Click on each and take note of the ‘FEED BASE URL’ as we will use this on the firewall to
retrieve the IoC list.
Sun Mgt Bonus Lab 9: Using Mine Meld for IoC Feed Aggregation 10
For access to live Palo Alto Networks lab boxes, go to: https://www.paloaltonetworks.com/services/education/cybersecurity-skills-practice-lab
6. Click ‘Add’
7. Create an EDL for both inboundfeedhc and inboundfeedmc
• Name: inboundfeedhc
• Type: IP List
• Source: <paste the FEED BASE URL from MineMeld>
• Repeat: Five Minute
8. Test Source URL
9. Repeat for inboundfeedmc
After creating both EDLs we can use them in firewall policy. Since both of these EDLs are
of IoCs configure policy to deny traffic. The example below we are denying Inside to Outside to
these EDL IPs.
Sun Mgt Bonus Lab 9: Using Mine Meld for IoC Feed Aggregation 11
For access to live Palo Alto Networks lab boxes, go to: https://www.paloaltonetworks.com/services/education/cybersecurity-skills-practice-lab
In the event you’d like to force a refresh of the List of Entries in an External Dynamic List
Hardware limitations
Check the number of external dynamic list entries used in policy to make sure you don’t
go over the firewall limit.
In PAN-OS 8.0, you can reference a total of 30 external dynamic lists with unique sources across
all security policy rules. In addition, external dynamic list entries (IP addresses, domain, and
Sun Mgt Bonus Lab 9: Using Mine Meld for IoC Feed Aggregation 12
For access to live Palo Alto Networks lab boxes, go to: https://www.paloaltonetworks.com/services/education/cybersecurity-skills-practice-lab
URLs) now only count toward the maximum number supported by the firewall if they belong to
lists referenced in Security policy rules you enforce on the firewall.
1. Select ObjectsExternal Dynamic List.
2. Click List Capacities in the bottom bar
Compare how many IP addresses, domains, and URLs are currently used in policy against
the total number of entries that the firewall supports for each list type. Since these values vary
from firewall to firewall, the List Capacities window is not available on Panorama.
Predefined IPs displays the number of IP addresses in the most recent Palo Alto Networks
Malicious IP Address Feeds saved to your firewall, even if they are not used in policy.
In addition to IoC you can use MineMeld for certain SaaS application. Due to the
dynamic IP nature of cloud-based applications, keeping updated on IP/URL changes and
incorporate them into firewall policy can be a daunting task. Fortunately, with MineMeld you
can add feeds from SaaS vendors that provide the information. In this example we will be
adding Office365 miners and feeds to our MineMeld instance.
MineMeld already come with Prototypes for each of the O365 services but you would
normally need to create a miner for each of these from those Prototypes, along with 3
processors and 3 outputs (one each for IPv4 addresses, IPv6 addresses and URLs respectfully).
To save you the hassle Palo Alto created a configuration you can import, simply download it
from https://paloaltonetworks.app.box.com/s/4ubmkgrq72a8mdd24j733ddqdgbkyvv4
NOTE: for a minimal config collecting all the IPv4s, IPv6 and URLs of all the O365 products
download this instead:
https://paloaltonetworks.box.com/s/gndwe5rzheg1ekwplxb4m3mrpcf5k41f
Sun Mgt Bonus Lab 9: Using Mine Meld for IoC Feed Aggregation 13
For access to live Palo Alto Networks lab boxes, go to: https://www.paloaltonetworks.com/services/education/cybersecurity-skills-practice-lab
7. Click Commit
Sun Mgt Bonus Lab 9: Using Mine Meld for IoC Feed Aggregation 14
For access to live Palo Alto Networks lab boxes, go to: https://www.paloaltonetworks.com/services/education/cybersecurity-skills-practice-lab
You should now have the miners, process, and output for office365 under Config section from
the ribbon.
After giving the MineMeld engine a few minutes to restart, click “Config” in the banner at the
top of the interface and then, click any of the nodes in the list.
Sun Mgt Bonus Lab 9: Using Mine Meld for IoC Feed Aggregation 15
For access to live Palo Alto Networks lab boxes, go to: https://www.paloaltonetworks.com/services/education/cybersecurity-skills-practice-lab
Similar to before, take note of the FEED BASE URL from the output mine (In this case
‘office365_IPv4s) so we can add the EDL to the firewall.
3. Click ‘Add’
4. Create an EDL for office365_IPv4s
• Name: office365_IPv4s
• Type: IP List
• Source: <paste the FEED BASE URL from MineMeld>
• Repeat: Five Minute
5. Test Source URL
Sun Mgt Bonus Lab 9: Using Mine Meld for IoC Feed Aggregation 16
For access to live Palo Alto Networks lab boxes, go to: https://www.paloaltonetworks.com/services/education/cybersecurity-skills-practice-lab
Now that we have EDLs we will modify/create our security policies. In the example below, we
are allowing our Office 365 apps for all known users in the Inside zone. The destination zone
has been set to Outside zone but with the IPv4 lists as destination addresses.
App-IDs that you may find detected during use of Office 365 (depending on the clients and
product sets being used)
activesync web-browsing
mapi-over-http webdav
ms-exchange ms-office365
ms-office365 office-live
ms-onedrive office-on-demand
rpc-over-http outlook-web-online
soap ms-lync-online
ssl ms-lync-online-apps-sharing
stun sharepoint-online
ms-lync-online-file-transfer
Sun Mgt Bonus Lab 9: Using Mine Meld for IoC Feed Aggregation 17
For access to live Palo Alto Networks lab boxes, go to: https://www.paloaltonetworks.com/services/education/cybersecurity-skills-practice-lab
a. Contact your Sun Management Account Rep to get pricing on a lab bundle. The
PA-220 and VM-50 appliances are excellent platforms for testing things such as
this and there are specific part numbers for lab equipment that are more heavily
discounted than the same appliance for use in production.
If you are unsure who your Account Rep is or do not have one yet, you can reach
out to [email protected] for assistance.
b. Reach out through the free Fuel Users Group (www.fuelusersgroup.org) which at
the time this lab is being written is offering limited free access to a virtual lab
environment, which they refer to as their “Virtual Test Lab,” in which you can
practice the steps outlined above. (Note: The Fuel Users Group may alter or
discontinue offering their “Virtual Test Lab” at any time)
c. For access to live Palo Alto Networks boxes for lab practice purposes
please go to:
https://www.paloaltonetworks.com/services/education/cybersecurity-
skills-practice-lab. This is a no charge service provided by Palo Alto
Networks.
If you feel Sun Management brings value to you and your organization with these
labs, please keep us in mind for other network and network security related
requirements. We are here to help you. Thank you for your business.
Resource Links
https://live.paloaltonetworks.com/t5/MineMeld-Articles/What-is-in-a-MineMeld-node/ta-
p/72046
https://live.paloaltonetworks.com/t5/MineMeld-Articles/How-to-Safely-Enable-access-to-
Office-365-using-MineMeld/ta-p/120280
https://www.paloaltonetworks.com/documentation/80/pan-os/newfeaturesguide/content-
inspection-features/external-dynamic-list-enhancements
https://live.paloaltonetworks.com/t5/MineMeld-Articles/Connecting-PAN-OS-to-MineMeld-
using-External-Dynamic-Lists/ta-p/190414
https://live.paloaltonetworks.com/t5/MineMeld-Articles/How-to-Safely-Enable-access-to-
Office-365-using-MineMeld/ta-p/120280