Cisco is honored to be a associate of the Black Hat NOC (Community Operations Middle), and this was our seventh 12 months supporting Black Hat Asia. Cisco is the Official Cellular Machine Administration, Malware Evaluation and DNS (Area Title Service) Supplier.
We work with different official suppliers to carry the {hardware}, software program and engineers to construct and safe the community, for our joint buyer: Black Hat.
- Arista: Community Tools
- Corelight: Community Analytics and Detection
- MyRepublic: Broadband
- NetWitness: Risk Detection & Response, Identification
- Palo Alto Networks: Community Safety Platform
The first mission within the NOC is community resilience. The companions additionally present built-in safety, visibility and automation, a SOC (Safety Operations Middle) contained in the NOC.
On screens outdoors the NOC had been displayed associate dashboards for the attendees to view the amount and safety of the community site visitors.
It All Began with Malware
Cisco joined the Black Hat NOC in 2016, when requested to supply automated malware evaluation with Thread Grid. The Cisco contributions to the community and safety operations developed, with the wants of the client, to incorporate extra elements of the Cisco Safety Cloud.
The NOC leaders allowed Cisco (and the opposite NOC companions) to herald extra software program to make our inner work extra environment friendly and have higher visibility; nevertheless, Cisco just isn’t the official supplier for Prolonged Detection & Response, Community Detection & Response or Collaboration.
- Breach Safety Suite
- Cisco XDR: Risk Searching / Risk Intelligence Enrichment / Govt dashboards / Automation with Webex
- Cisco XDR Analytics (Previously Safe Cloud Analytics / Stealthwatch Cloud): community site visitors visibility and menace detection
- Cisco Webex: Incident notification and workforce collaboration
The Cisco XDR Command Middle dashboard tiles made it straightforward to see the standing of every of the related Cisco Safety applied sciences, and the standing of ThousandEyes brokers.
When the companions deploy to every convention, we arrange a world class community and safety operations heart in three days. Our objective stays community up time and creating higher built-in visibility and automation. Black Hat has the choose of the safety business instruments and no firm can sponsor/purchase their means into the NOC. It’s invitation solely, with the intention of range in companions, and an expectation of full collaboration.
As a NOC workforce comprised of many applied sciences and corporations, we’re constantly innovating and integrating, to supply an general SOC cybersecurity structure answer. We sit up for persevering with the work with associate Palo Alto Networks, for additional automation at Black Hat USA 2024.
Beneath are the Cisco XDR integrations for Black Hat Asia, empowering analysts to analyze Indicators of Compromise (IOC) in a short time, with one search.
We admire alphaMountain.ai, Pulsedive and Recorded Future donating full licenses to Cisco, to be used within the Black Hat Asia 2024 NOC.
An instance of that is an investigation of a doubtlessly malicious exercise on the twond day of Coaching. An IP handle was recognized by NetWitness for doable geolocation leakage.
Investigation of the IP correlated the syslog sightings from the associate applied sciences within the NetWitness logs, with menace intelligence from Pulsedive, Recorded Future, alphaMountain and others.
Reviewing the DNS logs and the main points of the packet seize in each Corelight and NetWitness, it was confirmed no geolocation information was leaked and it was a part of a Coaching course. The exercise would have been blocked in a manufacturing surroundings.
A core built-in workflow within the Black Hat NOC is NetWitness and Corelight sending suspicious recordsdata to Safe Malware Analytics. Over 4,900 samples had been submitted.
The NOC analysts additionally used Malware Analytics to analyze suspicious domains, with out the chance of an infection. Reasonably than going to the web site on a company or Black Hat belongings, we had been in a position to work together with the web site within the glovebox, together with downloading and putting in the web site payload.
Detonating recordsdata or searching web sites in Safe Malware Analytics protects the analysts from unintentional an infection.
We noticed a collection of comparable (however completely different hash values) exploit kits downloaded on the primary day within the Enterprise Corridor. The downloads had been on the convention Wi-Fi and never in a Coaching course, so the occasion needed to be investigated to substantiate there was not an assault on the attendees. Working with the Corelight workforce, the NOC responders parsed the site visitors and confirmed it was a Seize the Flag occasion, which continued into the final day of the convention.
Risk Hunters’ Story, by Aditya Raghavan and Shaun Coulter
Within the Black Hat Asia 2024 NOC, Shaun staffed the morning shifts, and Aditya the afternoon shifts, as menace hunters targeted on the Cisco XDR and Safe Malware Analytics consoles. Mornings had been normally fairly chill. Nevertheless, and for some heretofore unknown (espresso associated?) motive, the exercise ramped up within the afternoon on most days, main Aditya to a spot of “concerned pleasure”, and Shaun to a spot of tormented jealousy :D. With dogged dedication each hunters spent their time reviewing alerts, actions, and carried out IOC scans utilizing XDR Examine. They reviewed submitted samples and community logs for indicators of intrusion or suspicious exercise.
Utilizing Safe Malware analytics, they dissected malware samples, analyzed phishing campaigns, and scrutinized community site visitors patterns for anomalies. Quite a few alerts flagged as spikes in site visitors from surprising sources, unusual locations and odd variants of malicious code popped up a number of instances, initiating thorough investigations. Usually, they traced the anomaly to a licensed Black Hat Coaching or Briefing supply and closed such instances as “Black Hat Constructive”; that means you wouldn’t permit this in your manufacturing community, however for Black Hat, it’s enterprise as common. Since Black Hat is a convention designed for studying about offensive safety, these malware samples are anticipated, and marked as such.
Fortunately or unthankfully, because the system tuning was accomplished, most alerts raised had been as above and anticipated or really ‘close to misses’ – objects that warrant investigation however didn’t lengthen to impactful behaviours, as we had been in a position to cease them in time.
On the primary day of Briefings, as Shaun is dutifully poring by way of the console of Safe Malware Analytics, in walks Aditya to alleviate the shift. Greetings apart, Shaun rapidly pivots over excitedly “Brother, I wish to present you a few attention-grabbing issues.” Aditya’s curiosity is piqued, and Shaun opens a brand new dashboard exhibiting one of many just lately launched options of Cisco XDR – MITRE ATT&CK ® Protection Map.
This new functionality rapidly shows all of the ways and methods within the MITRE ATT&CK® matrix for which Cisco XDR has detections/protection. Along with the XDR Native, detections from Safe Endpoint and Safe Malware Analytics are additionally used to derive the protection map making it a holistic view. This view permits the consumer to visualise the detections of XDR natively, in addition to the built-in options and determine the scope of protection and importantly, map out the gaps for future consideration. Due to the Cisco Talos workforce, all options throughout the Cisco Breach Safety Suite are mapped right this moment and this could be rolled out to incorporate different suites and options, together with 3rd social gathering integrations, quickly.
As our menace hunters geek out on the behind-the-scenes stuff on XDR, Jessica politely calls out “Adi. Shaun. Guys, there may be some new exercise on Umbrella. Are you able to look into it?” Nudged again to actuality, our menace hunters get to work – discovering needles within the stack of needles at Black Hat because it was rightly put by Grifter! Speaking about that, the brand new exercise seems to be a question for a site labeled as a Command & Management (C&C) area. Let’s dig into it.
A fast look into Umbrella Exercise Search reveals the most recent site visitors exercise matching the C&C class that was allowed. Increasing the main points pane, we are able to see the area being queried and the id of the endpoint issuing the question which seems to be from the ‘Hacking Enterprises 2024 Crimson Group’. That could be a professional Coaching class at Black Hat Asia 2024. We pivot over to Umbrella Examine and see the rationale for this area being categorized as C&C and its indicators.
Let’s head over to XDR and question this observable in opposition to all of the built-in options for extra intel. We rapidly get a visible related graph and tabulated occasions on all of the related intel. The combination with NetWitness Logs supplies us with occasions associated to that area, in addition to populating the graph with these relationships, together with the Umbrella occasion which was the supply for this hunt.
Wanting on the proof, this turned out to be one other needle! Nothing untoward right here, we labeled this as a ‘Black Hat Constructive’ and moved on. Because the afternoon shift winds down, the workforce is discussing potential locations for dinner and there may be all the time dessert to sit up for on the finish. Aditya and Ryan had been pining for wealthy ice cream and House Greatest Dessert seems to be the correct answer for the ask. Within the NOC, the correct answer is nearly all the time teamwork with all our companions.
One such occasion was when a Corelight hunter picked up a spike of site visitors to uncommon locations. These seem like DNS queries to a bunch of C&C domains. We rapidly delve into Umbrella exhibiting us all of the domains being queried in a brief window and most of them being Malware and/or C&C categorized. This seems to be a system both being compromised or somebody deliberately doing a take a look at / recon for these domains.
Let’s examine a few of these domains in XDR. We are able to see quite a lot of pink icons on this visualization! In reality, each queried area is classed as Malicious and identified to host different malicious content material. This doesn’t look anticipated for certain and that places the intentional take a look at / recon principle to relaxation rapidly. Ben Reardon, the hunter from Corelight, places it succinctly “This field is pwned six methods to Sunday!” What else can we discover about this method then?
Wanting on the DHCP logs for the IP handle, the Corelight hunter was in a position to pinpoint the system MAC handle and hostname, which resembled a reputation. A brief Google search later, we’ve a possible system proprietor and the truth that he was delivering a session at Black Hat in one of many rooms subsequent door! A brief dialog with the particular person after his session ensued, the place the NOC leads suggested the NOC’s findings on his compromised system. He was grateful for the discovering and reached out for extra context. This one turned out to be a ‘True Constructive.’
The next day, the workforce has zeroed in on Turkish meals for the night. Ryan halts Shaun as he departs on the finish of his shift and calls for his lodge identify and room quantity. “I’m gonna come knock at your door and wake you up tonight, man. I imply it. No day is simply too lengthy. I used to do my shifts on three hours of sleep. Now, let’s go!” Ryan is deadpan severe. That’s what we thought whereas investigating our subsequent potential malware discovering.
One other occasion on the Umbrella console involves our consideration and this time it’s a question for a site categorized as Malware. The supply endpoint is rapidly recognized from the Identification and Umbrella examine tells us this area is a part of the Malware block record. In a traditional manufacturing community, this could ideally be blocked.
Black Hat isn’t your regular manufacturing community, and it attracts all types of safety folks. And that’s precisely what it turned out to be this time. The Nationwide College of Singapore has a bunch organizing common seize the flag (CTF) occasions and is working an identical get-together at Black Hat. Go NUS Greyhats!
Actions involving malware what could be blocked on a company community should be allowed, throughout the confines of Black Hat Code of Conduct.
Community Observability with ThousandEyes, by Adam Kilgore and Patrick Yong
Deploying ThousandEyes at Black Hat is a rigorous course of involving quite a lot of {hardware} (some proven under), configuration, testing, troubleshooting, and working across the convention heart.
Along with our typical deployment duties, we applied a number of enhancements to the service. These enhancements included an overhaul of the dashboards to point out granular information for every convention room, alongside mixture information for the whole convention; and higher labeling and group of deployed brokers.
The ThousandEyes dashboard was projected on the massive display within the NOC, for alerting on any community points, previous to reviews from customers.
On the troubleshooting aspect, we improved our log evaluation and assortment methods and arrange centralized monitoring of wi-fi information. These efforts contributed to enhancements in visibility and agent uptime all through the convention.
In the course of the preliminary two days of Coaching classes at Black Hat, ThousandEyes brokers confirmed solely minor deviations from baseline because the Coaching classes got here on-line. Because the Coaching classes continued, efficiency was secure, with solely uncommon alerts for minor degraded throughput or average latency spikes. On Thursday, all of the two-day Coaching classes had been closed, and the convention shifted in the direction of Briefings, alongside two four-day Coaching classes that ran for the convention’s size. With begin of Briefings and opening the Enterprise Corridor, headcounts drastically elevated. ThousandEyes noticed degraded efficiency on the community, primarily within the massive convention rooms internet hosting the Briefings. The under picture reveals a take a look at consequence from the Hibiscus 3610 ballroom:
The community path above reveals heavy latency on the primary hyperlink to the default gateway, compounded by one other excessive latency hyperlink outdoors the convention community. A breakdown of connectivity for the above path is proven under:
The throughput quantity above is vital to this investigation. The Entry Factors (APs) for the Hibiscus 3610 ballroom had a mean throughput of round 174 Mbps. Reviewing AP logs, we discovered that 92 customers had been related to the identical AP from which the take a look at was run. Dividing the 174 Mbps by 92 provides a mean throughput in keeping with the 1.7 Mbps proven above, so the poor connectivity was pushed by oversaturation of consumer connections on this space.
The Hibiscus 3610 room and different brokers in a close-by hallway constantly had the worst connection among the many convention rooms, as proven by our agent polling outcomes.
Whereas there have been limitations within the quantity of bandwidth accessible for the convention normally, the info above suggests extra of the accessible AP and bandwidth assets must be allotted to the Hibiscus 3610 ballroom and adjoining hallways for future convention topologies, which was shared with our Community Tools associate.
Meraki Techniques Supervisor, by Paul Fidler and Connor Loughlin
Our eighth deployment of Meraki Techniques Supervisor because the official Cellular Units Administration platform went very easily, and we launched a brand new caching operation to replace iOS units on the native community, for pace and effectivity. Going into the occasion, we deliberate for the next varieties of units and functions:
- iPhone Lead Scanning Units
- iPads for Registration
- iPads for Session Scanning
We registered the units prematurely of the convention. Upon arrival, we turned every system on.
Then we ensured Location Companies enabled, all the time on.
As an alternative of utilizing a mass deployment know-how, like Apple’s Automated Machine Enrollment, the iOS units are “ready” utilizing Apple Configurator. This consists of importing a Wi-Fi profile to the units as a part of that course of. In Las Vegas, this Wi-Fi profile wasn’t set to auto be part of the Wi-Fi, leading to the necessity to manually change this on 1,000 units. Moreover, 200 units weren’t reset or ready, so we had these to reimage as properly.
Black Hat Asia was completely different. We took the teachings from Black Hat USA 2023 and coordinated with the contractor to arrange the units. Now, in the event you’ve ever used Apple Configurator, there’s a number of steps wanted to arrange a tool. Nevertheless, these could be mixed right into a Blueprint.
For Black Hat Asia this included:
- Wi-Fi profile
- Enrollment, together with supervision
- Whether or not to permit USB pairing
- Setup Assistant pane skipping
In Meraki Techniques Supervisor, we managed the purposes by the assigned use, designated by Tags. After we got here in on the primary morning of the Briefings, three iPhones wanted to be modified from lead scanning within the Enterprise Corridor, to Session Scanning for the Keynote, so the attendees may fill the corridor quicker. Reconfiguring was so simple as updating the Tags on every system. Moments later, they had been prepared for the brand new mission…which was essential because the Keynote room crammed and needed to go to an overflow room.
We additionally had been in a position to affirm the bodily location of every system if wiping was required because of loss or theft.
When it was time for the attendees to register, they simply displayed their QR code from their private cellphone, as acquired in e mail from Black Hat. Their badge was immediately printed, with all private particulars secured.
This goes with out saying, however the iOS units (Registration, Lead Seize and Session Scanning) do have entry to non-public data. To make sure the safety of the info, units are wiped on the finish of the convention, which could be accomplished remotely by way of Meraki Techniques Supervisor.
Content material Caching
One of many greatest issues affecting the iOS units in Black Hat USA 2023 was the rapid must each replace the iOS system’s OS because of a patch to repair a zero-day vulnerability and to replace the Black Hat iOS app on the units. There have been a whole bunch of units, so this was a problem for every to obtain and set up. So, I took the initiative into wanting into Apple’s Content material Caching service constructed into macOS.
Now, simply to be clear, this wasn’t caching EVERYTHING… Simply Apple App retailer updates and OS updates.
That is turned on withing System Setting and begins working instantly.
I’m not going to get into the weeds of setting this up, as a result of there’s a lot to plan for. However, I’d counsel that you simply begin right here. The setting I did change was:
Location and Jailbreak detection
One factor that we haven’t spoken about in a while is Jailbreak detection and Location. There are lots of parts that we get again from a tool, however two of them, Location and Jailbreak should be retrieved from a tool utilizing a supplemental utility: On this case, the Meraki Techniques Supervisor agent.
HOWEVER, these can solely be retrieved from the system if the applying is working within the background. If the system has been rebooted, or the applying terminated, then we don’t get something.
One of many different painful, however comprehensible, facets of MDM is that you could’t launch an utility distant on a cell system…. However you possibly can!
On each Android and iOS, there’s a functionality referred to as Kiosk or Single App mode: Use instances for this are usually unattended units, like in eating places, or scanning units like supply drivers, and so on. And when sending the command to the system to enter kiosk mode will launch the applying. You can even ship a command to take away kiosk mode from the system too. The beauty of this final level is that the applying stays in focus and open!
So, the opposite functionality that utilizing Meraki Techniques Supervisor provides us is the flexibility to schedule settings. Subsequently, we are able to activate kiosk mode in the course of the night time and take away it an hour later.
To make sure that this doesn’t affect the registration employees, we are able to go one step additional: after we’ve launched Meraki Techniques Supervisor, an hour later we are able to relaunch the registration utility, Swapcard Go.
Systematic ThousandEyes Agent Deployment
ThousandEyes has been a success at Black Hat. At an occasion the place understanding instantly the place points lie within the community and past to make sure an awesome convention is paramount, the visibility ThousandEyes provides is unimaginable. Provided that, and the complexity of the community right here, and on condition that we’ve a Mac Mini deployed for caching software program updates, as we’re utilizing Meraki Techniques Supervisor (SM) for different functions, I assumed I’d take the chance to deploy the ThousandEyes Agent utilizing SM.
The opposite motive is that, while we’ve a substantial quantity of cloud and enterprise brokers, we had no endpoint brokers deployed. Nevertheless, issues are by no means that straightforward with software program deployment, primarily as a result of you might want to provision / configure software program as soon as deployed. On cell units, that is simple, both utilizing settings payloads, or by utilizing Managed Appe Config to configure an app.
On desktop, utilizing MDM, we are able to usually use issues like Managed Plists to do the identical factor, however the TE agent does NOT help this. As soon as put in, we should name the agent with a string.
So, to attain all this, we are able to bundle the agent and command right into a bundle utilizing a command line utility on the Mac referred to as PKGBUILD (extra particulars right here).
I additionally used a information I’d written for the Meraki Group, accessible right here.
Info of notice:
The Postflight:
#!/bin/bash
# this identify will change with every model of the agent
installer -pkg /tmp/Endpoint Agent-x64-1.193.1.pkg -target /
/Functions/ThousandEyes Endpoint Agent.app/Contents/MacOS/te-agent –register “YOURUNIQUESTRING”
exit 0
The command to construct the bundle utilizing PKGBUILD
Extra particulars right here or watch the video.
Repurposing of Units for the subsequent present
We had been requested if there was something we may do to depart the units as they had been for the subsequent present. After cautious consideration, we determined that we may go away the units in a state that was amenable to everybody. The main requirement was leaving the Swapcard Go app on the system. However, because the app is provisioned for every present, it’s fairly the method to take away configuration after which re-add it….
So, the opposite factor to notice is the choices that we’ve when putting in (and eradicating) an utility on a managed iOS system:
Take away with MDM is the attention-grabbing one, because it permits us to, reasonably than WIPING the system on the finish of the present, to take away administration, together with any apps and settings, and their corresponding information.
The downside with that is that this was by no means a requirement firstly of the present. So, we now want a course of in a selected order to facilitate this…. As that is for less than a handful of units:
- Deprovision the app from units by unscoping the applying in Meraki Techniques Supervisor
- Wait to see this command has accomplished throughout all units
- Reprovision the app utilizing MDM once more, however with this being a brand new app set up, it is going to permit the OS to maintain the app in situ after an unenrollment
- Wait till accomplished
- Unenroll the system
Area Title Service Statistics, by Christian Clasen
Since 2018, we’ve been monitoring DNS stats on the Black Hat Asia conferences.
The historic DNS requests are within the chart under.
With over 18.2M DNS requests made, we had probably the most up to now at an Asia present. We made visibility developments on the earlier 12 months’s Asia convention. Previous to Asia 2023, we had been permitting attendees to make use of their chosen DNS resolvers over our assigned inner Umbrella Digital Home equipment. In coordination with Palo Alto Networks (the convention Firewall supplier), we started intercepting and redirecting DNS queries for different resolvers, to drive decision by way of the Umbrella gear. Whereas that is solely efficient for plain-text DNS queries and never encrypted protocols like DNS over HTTPS, it never-the-less dramatically elevated visibility as evidenced by the numbers within the accompanying charts.
The Exercise quantity view from Umbrella provides a top-level stage look of actions by class, which we are able to drill into for deeper menace looking. On pattern with the earlier Black Hat Asia occasions, the highest Safety classes had been Malware and Newly Seen Domains.
In a real-world surroundings, of the 18.2M requests that Umbrella noticed, over 2,000 of them would have been blocked by our default safety insurance policies. Nevertheless, since this can be a place for studying, we usually let every part fly.
We additionally monitor the Apps utilizing DNS, utilizing App Discovery.
- 2024: 4,327 apps
- 2023: 1,162 apps
- 2022: 2,286 apps
App Discovery in Umbrella provides us a fast snapshot of the cloud apps in use on the present. Not surprisingly, Generative AI (Synthetic Intelligence) has exploded over the earlier 12 months as a high utility.
Umbrella additionally identifies dangerous cloud purposes. Ought to the necessity come up, we are able to block any utility through DNS, akin to Generative AI apps, Wi-Fi Analyzers, or the rest that has suspicious undertones.
Once more, this isn’t one thing we’d usually do on our Basic Wi-Fi community, however there are exceptions. For instance, occasionally, an attendee will be taught a cool hack in one of many Black Hat programs or within the Arsenal lounge AND attempt to use stated hack on the convention itself. That’s clearly a ‘no-no’ and, in lots of instances, very unlawful. If issues go too far, we’ll take the suitable motion.
In the course of the convention NOC Report, the NOC leaders additionally report of the Prime Classes seen at Black Hat.
General, we’re immensely happy with the collaborative efforts made right here at Black Hat Asia, by each the Cisco workforce and all of the companions within the NOC.
Black Hat USA will likely be in August 2024, in Las Vegas. Christian Clasen will lead the Cisco workforce within the NOC, so comply with his weblog to see if what occurs in Vegas, stays in Vegas.
Acknowledgments
Thanks to the Cisco NOC workforce:
- Cisco Safety: Christian Clasen, Shaun Coulter, Aditya Raghavan, Adam Kilgore, Patrick Yong and Ryan Maclennan
- Meraki Techniques Supervisor: Paul Fidler and Connor Loughlin
- Extra Help and Experience: Adi Sankar, Robert Harris, Jordan Chapian, Junsong Zhao, Vadim Ivlev and Ajit Thyagarajan
Additionally, to our NOC companions NetWitness (particularly Iain Davidson and Alessandro Zatti), Palo Alto Networks (particularly James Holland and Jason Reverri), Corelight (particularly Mark Overholser and Eldon Koyle), Arista Networks (particularly Jonathan Smith), MyRepublic and the whole Black Hat / Informa Tech employees (particularly Grifter ‘Neil Wyler’, Bart Stump, Steve Fink, James Pope, Michael Spicer, Jess Jung and Steve Oldenbourg).
About Black Hat
Black Hat is the cybersecurity business’s most established and in-depth safety occasion collection. Based in 1997, these annual, multi-day occasions present attendees with the most recent in cybersecurity analysis, improvement, and tendencies. Pushed by the wants of the neighborhood, Black Hat occasions showcase content material instantly from the neighborhood by way of Briefings displays, Trainings programs, Summits, and extra. Because the occasion collection the place all profession ranges and tutorial disciplines convene to collaborate, community, and talk about the cybersecurity matters that matter most to them, attendees can discover Black Hat occasions in the US, Canada, Europe, Center East and Africa, and Asia. For extra data, please go to www.blackhat.com. See the press launch for Black Hat Asia 2024.
We’d love to listen to what you suppose. Ask a Query, Remark Beneath, and Keep Related with Cisco Safety on social!
Cisco Safety Social Channels
Share:
