Toolkits for Automating Malware Analysis

Automating some aspects of malware analysis is critical for organizations that process large numbers of malicious programs. Such automation allows analysts to focus on the tasks that require human insights. There are several free toolkits you can use as the starting point for building your own automated malware analysis lab.

• Jim Clausing documented his approach to building an automated behavioral malware analysis environment (PDF), building upon the Truman framework. Jim also provided several scripts to make it easier to make use of this methodology. 

• Christian Wojner documented his approach to building a customized malware analysis system, describing the methodology employed at CERT.at. You can download the associated toolkit called Minibis.

There are several other toolkits you may find useful for automating aspects of behavioral malware analysis:

• Cuckoo by Claudio Guarnieri is an open-source toolkit you can install locally for analyzing malicious files.

• Zero Wine by Joxean Koret is a full-featured tool for dynamically analyzing the behavior of Windows malware by running it within the WINE emulator on Linux.

• Buster Sandbox Analyzer by Buster is a wrapper around the Sandboxie tool for Windows, which helps you examine the key actions of applications executed by Sandboxie in your lab.

• Malheur by Konrad Rieck is a very promising tool for analyzing the volumes of data collected by behavioral sandboxes.

• REMnux by yours truly is a lightweight Linux distribution for assisting malware analysts in reverse-engineering malicious software.
• 
  
• SIFT from Sans

Blocklists of Suspected Malicious IPs and URLs

• ATLAS from Arbor Networks: Free; registration required by contacting Arbor
• Blackweb Project: Free; optimized for Squid
• CLEAN-MX Realtime Database: Free; XML output available
• CriticalStack Intel Marketplace: Free; registration required; optimized for Bro
• Certly Guard: Free tier available; registration required; query database via API
• CYMRU Bogon List: Free
• DShield Blocklist: Free
• FireHOL IP Lists: Free; combines several blacklists from other sources
• Google Safe Browsing API: Free; programmatic access; restrictions apply
• hpHosts File: Free; limited automation on request
• Malc0de Database: Free
• Malware Domain Blocklist: Free for non-commercial use
• MalwareDomainList.com Hosts List: Free
• Malware Patrol’s Malware Block Lists: Free for non-commercial use
• MalwareURL List: Commercial service; free licensing options may be available
• OpenPhish: Phishing sites; free for non-commercial use
• PhishTank Phish Archive: Free; query database via API
• Project Honey Pot’s Directory of Malicious IPs: Free; registration required to view more than 25 IPs
• Scumware.org: Free
• Shadowserver IP and URL Reports: Free; registration and approval required
• Squidblacklist.org: Paid and free options available
• ZeuS Tracker Blocklist and URLs: Free
• www.BlockList.de: Free

The lists differ in format, goals, and data-collection methodology. Be sure to read about the list before making use of it.

ShoDan Searches for Pentesting

Shodan provides a public API that allows other tools to access all of Shodan's data. Integrations are available for Nmap, Metasploit, Maltego, FOCA, Chrome, Firefox and many more.

API : https://developer.shodan.io/api

https://shodan.readthedocs.io/en/latest/

Shodan is a search engine but Instead of searching through content intentionally served up and delivered to web browsers, Shodan allows us to search for Internet-connected devices. Shodan uses distributed scanners throughout the world to randomly select target IP addresses and identify listening TCP and UDP ports. Listening ports are further enumerated to gather protocol banners, web pages, and other service data. All of this data is then added to an enormous, searchable database that describes the "what" of Internet devices.

Shodan is a search engine which does not index web sites or web contents, but vulnerable devices on the internet. To set up this index an to keep it up to date, Shodan uses at least 16 scanners with different AS numbers and different physical locations.

Shodan's search feature is powerful, allowing us to specify generic terms such as "camera" or even a specific part number such as "WVC80N" and quickly identify the devices that match.

example: camera / dlink / apache / mysql / vnc etc

From there you can pivot to a few key areas in the results. Starting on the left sidebar, we see a good amount of summary data:

Results map
Top services (Ports)
Top organizations (ISPs)
Top operating systems
Top products (Software name)

Then in the main section we get the full results list, including:

IP address
Hostname
ISP
When the entry was added to the database
The country it’s located in
The banner itself

Then, for even more information you can click details, which takes you into that host itself:

[ NOTE: When in details mode for a given host, the URL changes from the search structure to the following: https://www.shodan.io/host/194.69.36.22 ]

Here you see the data about the host on the left, the list of ports that were found at the top right, and then the individual port details and banners from each port as you go down the page.

Here we can see how to use the web interface for effective device searches, as well as tips to use Shodan in your next penetration test.


 Default Search Behavior
------ ------- ----------- -----
By default, Shodan's website search feature will use a search term as an exact expression in a string match. Shodan does not do incomplete word matching (e.g. "WVC80" will not return matches against "WVC80N"), and will treat multiple words as a logical AND expression. Common words (a, and, by, the, is, on, it) are ignored.

The basic search will perform string matching against server banner information without searching through additional protocol metadata that is also gathered about the discovered devices.

The Shodan documentation doesn't disclose exactly what protocol data is used in the default search, but empirical analysis indicates that it includes at least the following:

HTTP header information
HTTPS header and certificate information
Several gaming server banners (Steam's A2S, Minecraft, and more)
FTP banners
NetBIOS server banner
SSH header and server key data
Telnet banner
SMTP banner
NTP banner
SIP/VoIP banner
DNS server configuration settings
And more!

Metadata about a service is not searched by default. This list includes:

HTML title tag content
HTML header and body content
Physical location (via IP geolocation)
Autonomous System Number (ASN)
Internet Service Provider (by name, such as "Verizon Wireless")

 Shodan Search Operators / Filters
----- -------- ----------- --------- -----
To perform more advanced searches using Shodan, we can apply search operators. Search operators are only available to registered users. It's free to create an account, which will also give you an API key for use with Shodan's command-line tool.

Once you are logged-in, you can apply additional search modifiers to focus your search. Search operators / Filters include:

title: Search the content scraped from the HTML tag
html: Search the full HTML content of the returned page
product: Search the name of the software or product identified in the banner.  filters by technology (es. MySQL, Apache, IIS, Nginx);
net: Search a given netblock (example: 204.51.94.79/18)
version: Search the version of the product
port: Search for a specific port or ports
os: Search for a specific operating system name
country: find devices in a particular country (2-letter code)
city: find devices in a particular city
geo: you can pass it coordinates . filters by geographic coordinates;
hostname: find values that match the hostname
net: search based on an IP or /x CIDR
before/after: find results within a timeframe
org: with specific organization name

Some filters allow multiple values, such as "postal:97201,97202".

[ NOTE: You can drop the quotes sometimes, on some queries, but you often need them. I recommend you just use them all the time, because that always works. ]

 Examples:

port:22

product:"MySQL" country:"US"

country:"US" product:"MySQL" os:"Windows XP" version:"5.1.71-community"

city:"Bangalore" org:"google"

Find Apache servers in San Francisco: apache city:"San Francisco"

Find Nginx servers in Germany: nginx country:"DE"

Find GWS (Google Web Server) servers: "Server: gws" hostname:"google"

Find Cisco devices on a particular subnet: cisco net:"211.19.143.0/24"

Apache city:"San Francisco" port:"8080" product:"Apache Tomcat/Coyote JSP engine"

Juniper city:"Bangalore" org:"google"

Pulse Secure city:"Bangalore" org:"google"

city:"Bangalore" port:"8080" org:"google"

Default passwords country:IN

port:9600 response code

hostname: twitter.com

apache os:linux

openssh port:22

apache before:1/01/2014
nginx after:1/01/2014



By default, multiple search terms are treated as Boolean AND expressions. You can also negate a particular prefix with the "!" character at the beginning of the search operator.

For example, to search for machines running Outlook Web Access on ports other than 80 and 443, you can combine the title and port operators as follows:

title:"outlook Web Access" !port:443,80

 Applying Shodan in your Pen Test
-------- ---------- ---------------
It's easy to disregard Shodan as offering functionality to find vulnerable devices: an opportunistic attack tool. However, to do so is to overlook the benefits that Shodan can offer you and your customers in a penetration test.

Answering Questions About Similar Vulnerabilities, When putting together a report for a customer, I try to answer the inevitable question "How many others are similarly vulnerable?" Sometimes this question is in an attempt to justify a vulnerable configuration as commonplace or industry standard, or as a defensive mechanism for explaining why they continue to run Outlook Web Access on an IIS 5.0 server.

 Adding this level of detail to a penetration test report can help your customer to better understand the nature of the risk in the context of other similar configurations.

Search query: Microsoft-IIS/4.0 title:"outlook web"

 Scoping Targets by Network
--------- ---------- -------
Shodan can quickly disclose information about target devices scoped to a specific range of IP addresses. This can be useful for helping to get a quick understanding of your customer's assets and the services on those assets as known to Shodan.

For example, this author's office Internet access uses IP addresses in 100.10.23.0/24 block through Verizon FIOS. I can ask Shodan how many people with IP addresses in my network also have their routers available for remote authentication and access. Apparently, it's far too many.

Search query: net:100.10.23.0/24 unauthorized
net:144.36.222.0/23 unauthorized city:"Bangalore" org:"google"

 Scoping Targets Without IP Ranges
---------- ---------- -------------
Sometimes the point of contact you are working with to scope your penetration test might not be aware of the company's entire web presence. By searching for identifying features of the website (such as the copyright notice), you may be able to find lesser-known sites for a given organization.
As a penetration tester, identifying targets that are owned by an organization that they don't know allows you to clearly demonstrate your value and usefulness as a security analyst.

For example, a search for html:"eBay Inc. All Rights Reserved" shows a small number of sites (eBay has excluded a lot of their web properties from Shodan) that may not be as well known:

Search query: html:"eBay Inc. All Right Reserved"

If your target is large enough to have Regional Internet Registry allocations (where the WHOIS information reflects the organization name), you can combine negative searches to exclude the known ranges with the html filter (searching for copyright or other unique strings) or the "org" filter.

Search query: title:"eBay Deals" -org:"EBAY"

Using the power of Shodan and some creative thinking, you can provide additional value to your penetration tests. Use some of these ideas in your next pen test and see if you can find some targets that were supposed be in scope.

 USE CASES
   -----------
You can use the “Explore” button on the main Shodan site to look at common searches and results, which are illuminating. You’ll find things like:

Webcams
SCADA
Traffic lights
Routers
Default passwords Etc.

 COMBINING FILTERS
  ------- --------- ---------
To combine filters, simply keep adding them on. You can also do this by clicking filters in the left sidebar for a given result set. So if you want to search for Nginx servers in San Francisco, that are running on port 8080, that are also running Tomcat, you could do the following:

Apache city:"San Francisco" port:"8080" product:"Apache Tomcat/Coyote JSP engine"

Here are a few other cool things you can do with the service.

Data Export: You can export your results in various formats using the top menu after you’ve performed a search. Browser Search: You can configure your browser to search Shodan when you search from the URL bar. Shodan Free Account: You should create and log in to your free account when you search, as the interface is pretty nerfed if you don’t, e.g. not being able to see host information, etc.

Premium Accounts: A premium account is a one-time payment of $45 and it gives you increased access to the API. Full details and docs are available at https://developer.shodan.io.


https://exploits.shodan.io/welcome
https://cli.shodan.io/

https://shodanio.wordpress.com/2014/12/01/using-shodan-from-the-command-line/

https://shodanio.wordpress.com/2013/12/05/shodan-fingerprints-search-by-product-or-specific-version/

https://shodanio.wordpress.com/2013/08/25/announcing-the-new-shodan-exploits/

http://wiki.ipfire.org/en/configuration/firewall/blockshodan

https://www.hackers-arise.com/single-post/2016/06/22/Using-Shodan-The-Worlds-Most-Dangerous-Search-Engine

https://www.exploit-db.com/docs/33859.pdf

https://www.youtube.com/watch?v=U-_pc2cMbuA

https://www.youtube.com/watch?v=BV01xyAdeOI

https://www.shodan.io/about/products

https://learnhackin.wordpress.com/2016/08/29/how-to-find-vulnerable-webcams-shodan-tutorial/

https://en.wikipedia.org/wiki/Shodan_(website)

https://www.defcon.org/images/defcon-18/dc-18-presentations/Schearer/DEFCON-18-Schearer-SHODAN.pdf

Alienvault Threat Intelligence and Pulse creation

OTX Reputation Monitor Alert – free service What is AlienVault’s OTX Reputation Monitor Alert? Leveraging the world’s only open and collaborative IP reputation database, AlienVault’s OTX Reputation Monitor Alert monitors the reputation of your assets (public IPs and domains) and emails you notifications whenever there are changes.

What threats does it uncover? Malware Infections Spamming Hosts Malicious Activity Potential Breaches Compromised Websites Hosts being used for Botnets 8.

Where are we monitoring for you? These events will trigger an alert:

     OTX IP/Domain Match
     Presence in Pastebin/Pastie
    Presence on a DNS Blacklist
    DNS Registration Update – informational only
    SSL Certificate Update – informational only

How does the service work :

 1. Sign up via our OTX portal.
 2. Register your organization’s public IPs and domains.
 3. When there’s a match on one of our alert types, we’ll email you an alert with
        more information and remediation advice.
 4. You’ll also receive our monthly threat intelligence newsletter. Registration takes
    just a few minutes…

What is Open Threat Exchange (OTX) :

   An open and collaborative initiative for security professionals to connect with their
    peers, find free tools for security monitoring,  and learn about the latest threats
   and defensive tactics from security researchers.

   Open source threat intelligence projects and services including OSSIM and
       OTX Reputation Monitor Alert.

   Centralized place for these rich resources:
   OTX Projects
  OTX Blog
  OTX Forums
  OTX Learning Center

What is Threat Intelligence :

 Information about malicious actors.

 • Helps you make better decisions about defense
 • Examples: IP addresses, Domains, URL’s, File Hashes, TTP’s, victim’s industries, countries..

Standards & Tools :

 • IODEF: Incident Object Description Exchange Format
 • MITRE: – STIX: Structured Threat Information eXpression –
            TAXXII: Trusted Automated eXchange of Indicator Information – MAEC, CAPEC, CyBOX
 • CIF: Collective Intelligence Framework

What is IP Reputation : IP Reputation is a summary of the past behavior activity detected on an IP An IP with reputation information add context when a network connection is observed.

What is an IP Reputation engine : An IP Reputation engine is a system to classify and score large sets of IPs, in low or high reputation.

Domain Reputation & Heuristic Domain :

The industry’s reticence to share information about attack vectors gives the adversary a huge advantage. Using Threat Intelligence we can reduce this advantage and enable preventative response.
We will guide you through the different standards (OpenIOC, STIX, MAEC, OTX, IODEF…) to describe and share cyber intelligence, as well as Open Source Frameworks such as CIF (Collective Intelligence Framework) that allows you to combine different threat sources.

One of the biggest problems with Threat Intelligence is finding out how to take advantage of the data you have to actually improve the detection/prevention capabilities in your environment. We will describe how to leverage Threat Intelligence to detect threats and provide defenses, and we will focus on how to use Open Source Tools (Suricata, OSSIM, OSSEC, Bro, Yara…) to get the most of your Threat Intelligence.

(check sandbox and CIF integration)

Delivery & Attack : means a type of Alarm which generates alert about “an attempt to deliver an exploit” to the monitored system.


Open Threat Exchange(OTX) :

                        OTX allows anyone in the security community to actively discuss, research, validate, and share the latest threat data, trends, and techniques.

The OTX community reports on and receives threat data in the form of pulses. An OTX pulse consists of one or more indicators of compromise (IOCs) that constitute a threat or define a sequence of actions that could be used to carry out attacks on networks devices and computers. OTX pulses also provide information on the reliability of threat information, who reported a threat, and other details of threat investigations.                

All OTX members receive pulse information through their OTX Activity feed, as well as receive updates about pulses through email. This information appears as soon as you open an OTX account. OTX data can be used to enhance the threat detection capabilities, using SIEM tools.

You can sign up and set up an OTX account based on an email address, or you can use your existing Twitter or Google+ account.

Go to https://otx.alienvault.com

After you got confirmation mail go to OTX Home page https://otx.alienvault.com/dashboard/new/ where you will see all current pulse activity.

After logging in to the OTX user interface, you can begin examining the current threat activity being reported by various contributing members and groups. OTX lets you browse and search all current threats being reported, track updates on specific threat pulses, and follow or subscribe to the
contributions of specific pulses, members, and groups.

he main, middle portion of the display provides an activity feed or stream of OTX pulses. At the top of this stream, the OTX user interface provides three main tab selections to display results for Pulses, Activity, and Suggested Edits.

Besides the short results displayed for each pulse, you can click on a specific pulse in the results list, to view more detailed information about the pulse, additional attributes, and indicators of compromise (IOCs) for the pulse.

In this panel, you can click the Find a Group link or the Plus ( ) button to find and join other groups of OTX users that you may share interests with. You can choose options to Join an Existing Group, Join a Private Group, or Create a Group.

Note: Joining groups means you will get alerts when pulses are added, and you can retrieve pulses more easily into a SIEM solution using the OTX DirectConnect API. Some groups are private, so pulses contributed from those groups can only be viewed if you are a member.

In the left-side panel, you can click the Profile ( ) button to view pulses sorted, in order, by different categories. By default, the result list shows pulses across all categories. From the tab menu bar, you can select to view pulses by categories such as by Groups (N), Followers (N), Following (N), Subscribers (N), and Subscribing (N).

The lower portion of the pulse details display provides a comment section and information on Indicators of Compromise (IOCs) for the selected pulse. Depending on the particular indicator of compromise, the Indicator Details page can be very simple or it can include a great deal of information and research, based on how much information is available or known about the indicator at any given time.

IP Reputation Data Sources :

       IP Reputation receives data from a variety of sources, including the following:
       Hacker forums
       Open-source intelligence — Public and private security research organizations.

Browsing OTX :

            You can search by a number of fields within pulses. In the created and modified fields, search criteria is specified in the format <number ymd. For example,
<1w would search all pulses within the last week.

adversary:
author:
created:
description:
indicator:
industry:
modified:
name:
reference:
tag:
tip:

Subscribing to a publicly-created pulse allows automatic export of its raw data to the security tools you use to monitor security in your environment (provided you have configured your security tools to connect with OTX). 

Note: When you unsubscribe from a pulse, you still receive information about the threat in your OTX pulse activity feed, but no raw data is pulled into your security tools for correlation and generation of alarms. You might consider unsubscribing to a pulse if it is creating too much noise and generating too many false positive alarms in USM Appliance or whatever security monitoring tools your organization is using.

You may also subscribe to or follow public OTX contributors, in addition to subscribing to individual pulses.

Subscribing to an OTX member or contributor directs OTX to send all of the contributor’s pulses or IOCs to the tools used to monitor security in your environment.

Pulse and IOC contributions from an OTX member or contributor you subscribe to automatically appear in your OTX pulse activity feed, and you also receive emails every time they update one of their pulses or when they create a new pulse.

Note: After subscribing to a pulse, you can also always unsubscribe from the pulse, if it is creating too much noise and generating too many false positive alarms in USM Appliance or whatever security monitoring tools your organization is using. When you unsubscribe from a pulse, you still receive information about the threat in your pulse activity feed, but no raw data is pulled into your security tools for correlation and generation of alarms.

Contributing to OTX :

         Note: You can choose to stop sharing data with OTX at any time by going to the USM OTX Configuration page (Configuration > Open Threat Exchange).

Creating and Updating Pulses :

When you choose to create and contribute pulses to OTX, you can use a number of different methods to do so:

  * Use the OTX extraction wizard to pull IOCs from your favorite sources. These can be blogs,
emails, a PDF file, log files, or any other malware sources—any file that has a textual description of a threat. You can also import Open IOC 1.x and STIX files.

 * Manually add indicators of compromise to create a pulse.
 * Copy and paste indicators into the detail of a new pulse.
 * Clone an existing pulse possessing the characteristics of a pulse you want to
 create, and then edit the cloned pulse to create a new pulse.
 * Open an existing pulse you’ve created and add indicators, either manually or using
            the AlienVault Indicator Extractor.

  1. From the OTX main menu, select Create Pulse.

  2. In the Extract from Source (AlienVault Indicator Extractor) section of the Create New Pulse page, do one of the following, depending on the type of indicator you want to define for the new pulse:
          Type the URL of a website or blog.
           Drag and drop a text file (for example, a PDF, text, plain text log,
                                               STIX, or OpenIOC file).
           Paste the text describing an indicator.

  3. Click Next.
       OTX processes the request and displays the new pulse page with the newly Included indicators.

  4. If OTX found any excluded IOCs, review the list of Excluded IOCs tab.

 This tab includes items that OTX determined were unlikely to pose threats.
However, it is good practice to scan the list anyway, in case you see something about which you do not agree.

  5. If you see something suspicious on the list, transfer it to the list of Included IOCs.

  6. Click Next.
 OTX displays a final Create New Pulse page to include other details describing the new pulse you want contribute. The specific indicator you added on the previous page, and its type (for example, domain), appear on the right side of the page.

   7. Identify the pulse and complete the pulse description with the following information:
        TLP — Indicate the Traffic Light Protocol (TLP) for the threat by expanding the TLP list. The TLP consists of designations used to help ensure that sensitive information is shared with the correct audience. Its four colors indicate different degrees of sensitivity and the corresponding sharing considerations to be applied by the recipient(s). For guidance, see https://www.us-cert.gov/tlp.

Name — Give the pulse a concise name that uniquely characterizes the threat. This could consist of where the threat was found or what type of malware it represents, for example, “New PoSeidon spotted”.

Description — Describe the pulse in terms of where you found it, the type of threat it poses, and any other facts that may link it to other threat indicators.

Private — Indicate whether or not you want to share the pulse with others or make it private. (Private means that you do not want to share the pulse with others, because you need to conduct more research.)

Tags — If your IOC is a URL, OTX creates relevant tags based on its analysis of the URL. You can review any of these tags, and delete them, or you can add a new tag you feel is relevant.

Groups — Add groups to associate with pulse.

Industries — Specify primary industries targeted by threat.

Targeted Countries — Specify countries that have been targeted by threat.

8. After reviewing all your entries for the new pulse, click Submit.

 
Integrating OTX Data with External Security Monitoring Systems :

   To leverage OTX threat intelligence within your own security monitoring and management systems and tools, you can easily do that by taking advantage of the AlienVault OTX DirectConnect API and DirectConnect SDK. You can connect to the OTX API using DirectConnect Agents available for a number of specific products and third party tools.

The DirectConnect SDK provides support for development of DirectConnect agents or connectors for the following programming environments:
      Java
      Python
      Golang
    You can also always access the DirectConnect API using a command-line FTP/HTTP data transfer tool such as curl to access OTX threat intelligence information. For example:

curl https://otx.alienvault.com:443/api/v1/pulses/subscribed?page=1 -H "X-OTX-API-KEY a5bb241cd31fc1db69fcf0fd611161606061b9445e3758fd3e71d50e6477e12a"

2. The OTX user interface directs you to the AlienVault Labs SDK documentation page on GitHub for the specific language option you selected, for example, the OTX-Python-SDK resource page.

To access the Direct API SDK from the Settings Page :

1. From the OTX user interface Home page, click on the Settings ( ) icon and choose the
Settings menu option.

2. In the OTX key section of the page, click the Use the OTX API SDK link.

3. The OTX user interface directs you to the AlienVault Labs SDK documentation page on GitHub
which provides various support reference links for each of the different languages that are supported for the API.

Finding Unknown Malware – Step-By-Step

STEP 1: Prep Evidence/Data Reduction

• Carve and Reduce Evidence
- Gather Hash List from similar system (NSRL, md5deep)
- Carve/Extract all .exe and .dll files from unallocated space

  https://www.nsrl.nist.gov/
  http://www.forensicswiki.org/wiki/Md5deep

• foremost • sorter (exe directory) • bulk_extractor

• Prep Evidence
- Mount evidence image in Read-Only Mode
- Locate memory image you collected
- Optional: Convert hiberfil.sys (if it exists to raw memory image) using volatility


STEP 2: Anti-Virus Checks

Run the mounted drive through an anti-virus scanner with the latest updates. Anti-virus scanners employ hundreds of thousands of signatures that can quickly identify
well-known malware on a system. First, download the latest anti-virus signatures
and mount your evidence for analysis. Use a “deep” scan when available and
consider scanning your mounted drive with multiple anti-virus engines to take
advantage of their scanning and signature differences.

Get in the habit of scanning files exported from your images such as deleted files, data carving results, Sorter output, and email attachments. While anti-virus will not be effective on 0-day or unknown malware, it will easily find the low hanging fruit.

STEP 3: Indicators of Compromise Search

Using indicators of compromise (IOCs) is a very powerful technique to identify malware components on a compromised host. IOCs are implemented as a combination of boolean expressions that identify specific characteristics of malware. If these characteristics are found, then you may have a hit. An IOC should be general enough to find modified versions of the same malware, but specific enough to limit false positives.

There are two types of indicators: host-based (shown above), and network-based (similar to snort
signatures plus additional data).

The best IOCs are usually created by reversing malware and application behavioral analysis.
 
   Or
     FileName
     File Md5
     File Fuzzy Hash
   And
     File Path Contains \windows\system32\
     File Size is > 100kb and < 150kb
   Or
     Service Name = svchost
     " "    = srvavc
   And
     Malware is Packed
     Compile time is between 2-Dec and 10 Dec
   Or
     RegKey = Softwaare\Microsoft\Windows\CurrentVersion\Run
     RegKey = Softwaare\Microsoft\Windows\CurrentVersion\RunOnce

And
       Registry Value = winzf32

 What Works?
            OpenIOC Framework - openioc.org
            IOC Editor
            IOC Finder
            YARA Project
 

STEP 4: Automated Memory Analysis

    • Behavior Ruleset
- Code Injection Detection
- Process Image Path Verification

    • svchost outside system32 = Bad
- Process User Verification (SIDs)

    • dllhost running as admin = Bad
             - Process Handle Inspection

    • iexplore.exe opening cmd.exe = Bad
    • )!voqa.i4 = known Poison Ivy mutant
    • Verify Digital Signatures
- Only available during live analysis
- Executable, DLL, and driver sig checks
- Not signed?
    • Is it found in >75% of all processes?

What Works?
MANDIANT Redline
   www.mandiant.com/products/free_software/redline
Volatility Malfind
      http://code.google.com/p/volatility

STEP 5: Evidence of Persistence Malware
 
        Scheduled Tasks
Service Replacement
Service Creation
Auto-Start Registry keys
DLL Search Order Hijacking
Trojaned legitimate System Libraries
More Advanced - Local group policy, MS office Add-In, or BIOS Flashing

 Malware wants to hide, but it also wants to survive a reboot. Malware persistence is extremely common and is an excellent way to find hidden malware.
Persistence comes in many forms. The simplest mechanism is via scheduled tasks and the “at” command. Other popular persistence mechanisms include Windows Services and auto-start locations. Adversaries can run their malware as a new service or even replace an existing service.
There are numerous Windows Registry mechanisms to auto-start an executable at boot or login.Using a tool called autorunsc.exe will easily parse the autostart locations across scheduled tasks, services, and registry keys. While these are the most common, keep in mind there are more advanced techniques.

For example, the Mebromi malware even flashes the BIOS to persist.
Attacks of this nature are rare because even the simplest of techniques are effective,
allowing attackers to maintain persistence for long periods of time without being discovered.

 What Works?
  Autorunsc.exe from Microsoft sysinternals
   http://technet.microsoft.com/en-us/sysinternals/bb963902
 
STEP 6: Packing/Entropy Check 

 • Scan the file system or common locations for possible malware
- Indication of packing
- Entropy test
- Compiler and packing signatures identification
- Digital signature or signed driver checks

What Works?
MANDIANT Red-Curtain http://www.mandiant.com/resources/download/red-curtain
DensityScout http://cert.at/downloads/software/densityscout_en.html
Sigcheck - http://technet.microsoft.com/en-us/sysinternals/bb897441

STEP 7: Review Event Logs 

     Scheduled Tasks Log
Logon Events
Account Logon Events
Rogue Local Accounts
Suspicious Services
Clearing Event Logs

    What Works?
logparser - http://www.microsoft.com/download/en/details.aspx?id=24659
Event Log Explorer - http://eventlogxp.com
Log Parser Lizard - http://www.lizard-labs.net

STEP 8: Super Timeline Examination

Once you are down to about 10-20 candidates, it is a good time to identify where
those files show up in your timeline. The additional context of seeing other files in
close temporal proximity to your candidates allows you to identify false positives and
focus on those files most likely to be malicious.  In the above example, we see the
creation of the file winsvchost.exe in the C:\Windows\System32\ directory.
If this were one of your candidate files, you would clearly see artifacts that indicate a
spear phishing attack surrounding that file’s creation time.

Notably, an .XLS file was opened via email, winsvchost.exe was executed,an
auto-start persistence mechanism was created, and finally, a network socket was opened.

All within one second! Contextual clues in temporal proximity to the files
you are examining are quite useful in your overall case.

What Works? log2timeline found in SIFT Workstation
       http://computer-forensics.sans.org/community/downloads

     
STEP 9: By-Hand Memory Analysis

     * Identify rogue processes
               - name,path,parent,command line,start time,SIDs

     * Analyze process DLLs and handles

     * Review network artifacts
               - Suspicious ports, connections and processes

     * Look for evidence of code injection
            - Injected memory sections and process hollowing

     * Check for signs of a rootkit
            - SSDT, IDT,IRP and inline hooks

     * Dump suspicious process and drivers
           - Review strings, anti-virus scan, reverse-engineer

  Memory analysis is one of the most powerful tools for finding malware. Malware
has to run to be effective, creating a footprint that can often be easily discovered
via memory forensics. A standard analysis can be broken down into six major steps.
Some of these steps might be conducted during incident response, but using a
memory image gives deeper insight and overcomes any rootkit techniques that
malware uses to protect itself. Memory analysis tools are operating-system specific.
Since each tool gathers and displays information differently, use multiple tools to check your results.

    What Works?
         Volatility http://code.google.com/p/volatility
                  Mandiant Redline www.mandiant.com/products/free_software/redline


 
STEP 10: By-Hand 3rd Party Hash Lookups

      Hash lookups to eliminate known good files and identify known bad files is a useful technique when narrowing down potential malware. Bit9 FileAdvisor is a free search engine for querying Bit9’s application whitelisting database.

  It is available via online lookup, as well as via a downloadable utility (http://fileadvisor.bit9.com/services/wu/latest/FileAdvisor.msi).

 The National Software Reference Library also provides a robust set of known good hashes for use.

VirusTotal will scan a file through over 40 different A/V scanners to determine if any of the current signatures detect the malware. VirusTotal also allows its database to be searched via MD5 hashes, returning prior analyses for candidate files with the same MD5.


      What Works?
            VirusTotal www.virustotal.com and bit9 http://fileadvisor.bit9.com
                 NSRL Query http://nsrlquery.sourceforge.net

STEP 11: MFT Anomalies

A typical file system has hundreds of thousands of files. Each file has its own MFT
Record Number. Because of the way operating systems are installed, it’s normal to
see files under entire directory structures written to disk with largely sequential MFT Record Number values.

For example, above is a partial directory listing from a Windows NTFS partition’s %system32% directory, sorted by date.

Note that the MFT Record Number values are largely sequential and, with some exceptions, tend to align with the file creation times. As file systems are used over the years and new patches are applied causing files to be backed up and replaced, the ordering of these files by MFT Record Number values can break down. Surprisingly, this ordering remains intact enough on many systems, even after years of use, that we can use it to spot files of interest. This will not happen every time as
MFT entries are recycled fairly quickly, but in many cases an outlier can be identified.

STEP 12: File-Time Anomalies

        * Timestamp Anomalies
- $SI Time is before $FN Time
- Nanoseconds values are all zeroes

One of the ways to tell if file time backdating occurred on a windows machine is to examine the NTFS $Filename times compared to the times stored in $Standard Information. Tools such as timestomp allow hackers to backdate a file to an arbitrary time of their choosing.

Generally, hackers do this only to programs they are trying to hide in the system32 or similar system directories. Those directories and files would be a great place to start. Look to see if the $Filename (FN) creation time occurs after the $Standard Info creation time, as this often indicates an anomaly.

What Works?
          analyzeMFT.py found on SIFT Workstation and
                  www.integriography.com
               log2timeline found on SIFT Workstation

STEP 13: You Have Malware! Now What?

      • Hand it to Malware Analyst
         - FOR610 – RE Malware
         - Hand over sample, relevant configuration files, memory snapshot

      • Typical Output from Malware Analyst
         - Host-based indicators
         - Network-based indicators
         - Report on malware capabilities and purpose

 • You can now find additional systems compromised by the malware you found

Python Setup

1. Download Python :
                                  https://www.python.org/downloads/

2. Execute python-2.7.12.msi : 

                              Python installs to a directory with the version number embedded, e.g. Python version 2.7 will install at C:\Python27\, so that you can have multiple versions of Python on the same system without conflicts. Of course, only one interpreter can be the default application for Python file types.

It also does not automatically modify the PATH environment variable, so that you always have control over which copy of Python is run.

3. Typing the full path name for a Python interpreter each time quickly gets tedious, 
  so add the directories for your default Python version to the PATH. 
  Assuming that your Python installation is in C:\Python27\, add this to your PATH:

                C:\Python27;C:\Python27\Scripts

The second (Scripts) directory receives command files when certain packages are installed, so it is a very useful addition.

Install the tools and libraries described in the next section before you start building Python
 applications for real-world use. In particular, you should always install Setuptools, 
 as it makes it much easier for you to use other third-party Python libraries.

4. Setuptools + Pip

  The most crucial third-party Python software of all is Setuptools, 
which extends the packaging and installation facilities provided by the distutils in the 
standard library. Once you add Setuptools to your Python system you can download and 
install any compliant Python software product with a single command. It also enables you 
to add this network installation capability to your own Python software with very little work.

To obtain the latest version of Setuptools for Windows, run the Python script available here:

    https://bootstrap.pypa.io/ez_setup.py

The script will download the appropriate distribution file and install it for you.
Once installation is complete, you will find an '" easy_install " program in your Python Scripts sub-directory.

You’ll now have a new command available to you: easy_install. It is considered by 
many to be deprecated, so we will install its replacement: pip.

Pip allows for uninstallation of packages, and is actively maintained, unlike easy_install.

# python ez_setup.py

    For installing pip

Download get-pip script from the location

# https://bootstrap.pypa.io/get-pip.py

C:\Python27>python get-pip.py

 Verify a successful installation by type pip freeze. pip freeze displays the version 
number of all modules installed in your Python non-standard library.

C:\Python27>pip freeze Or pip list

freeze : Output installed packages in requirements format.
list   : List installed packages.

So there are two differences:

Output format, freeze gives us the standard requirement format that may be used later 
with pip install -r to install requirements from.

Output content, pip list include editable which pip freeze does not

Pip is using for install / uninstall / download new packages

 pip --help

https://pip.pypa.io/en/stable/reference/pip_install/

python
Python 2.7.9 (default, Mar  1 2015, 12:57:24)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
ctrl+D --- in linux  and
ctrl+Z --- in windows

C:> python --version

Building a Malware Analysis Toolkit

Examining the capabilities of malicious software allows your IT team to better assess the nature of a security incident. Here’s how to set up a controlled malware analysis lab—for free.

Step 1: Allocate physical or virtual systems for the analysis lab
Step 2: Isolate laboratory systems from the production environment
Step 3: Install behavioral analysis tools
Step 4: Install code-analysis tools
Step 5: Utilize online analysis tools
Step 6: Next Steps

When investigating the incident, the IT guys typically seeks to answer questions such as:

 * What actions can the malware specimen perform on the system?
 * How does it spread?
 * How, if at all, does it maintain contact with the attacker?

 These questions can all be answered by analyzing the offending malware in a
  controlled environment.

 A simple analysis toolkit, built from free and readily available software, can help you and your IT team develop the skills critical to responding to today’s security incidents. The steps below will help get you started. We’ll focus on malware analysis in a Windows environment, since that platform is particularly popular among malware authors.

Step 1: Allocate physical or virtual systems for the analysis lab:

    A common approach to examining malicious software involves infecting a system with the malware specimen and then using the appropriate monitoring tools to observe how it behaves. This requires a laboratory system you can infect without affecting your production environment.

    The most popular and flexible way to set up such a lab system involves virtualization software, which allows you to use a single physical computer for hosting multiple virtual systems, each running a potentially different operating system.

Running multiple virtual systems simultaneously on a single physical computer is useful for analyzing malware that seeks to interact with other systems, perhaps for leaking data, obtaining instructions from the attacker, or upgrading itself. Virtualization makes it easy to set up and use such systems without procuring numerous physical boxes.

Another useful feature of many virtualization tools is the ability to take instantaneous snapshots of the laboratory system. This way, you can record the state of the system before you infect it, and revert to the pristine environment with a click of a button at the end of your analysis.

If using virtualization software, install as much RAM into the physical system as you can, as the availability of memory is arguably the most important performance factor for virtualization tools. In addition, having a large hard drive will allow you to host many virtual machines, whose virtual file systems typically are stored as files on the physical system’s hard drive.

    Because malware may detect that it’s running in a virtualized environment, some analysts prefer to rely on physical, rather than virtual, machines for implementing laboratory systems. Your old and unused PCs or servers can make excellent systems for your malware-analysis lab, which usually doesn’t need high-performing CPUs or highly redundant hardware components.

    To allow malware to reach its full potential in the lab, laboratory systems typically are networked with each other.  This helps you observe the malicious program’s network interactions. If using physical systems, you can connect them with each other using an inexpensive hub or a switch.

Step 2: Isolate laboratory systems from the production environment:

    You must take precautions to isolate the malware-analysis lab from the production network, to mitigate the risk that a malicious program will escape. You can separate the laboratory network from production using a firewall. Better yet, don’t connect laboratory and production networks at all,
 to avoid firewall configuration issues that might allow malware to bypass filtering restrictions.

    If your laboratory network is strongly isolated, you can use removable media to bring tools and malware into the lab. Consider using write-once media, such as DVDs , to prevent malicious software from escaping the lab’s confines by writing itself to a writable removable disk. A more convenient option is a USB key that includes a physical write-protect switch.

Some malware-analysis scenarios benefit from the lab being connected to the internet. Avoid using the production network for such connectivity. If possible, provision a separate, and usually inexpensive, internet connection, perhaps by dedicating a DSL or Cable Modem line to this purpose.
Avoid keeping the lab connected to the internet all the time to minimize the chance of malware in your lab attacking someone else’s system on the internet.

If virtualizing your lab, be sure to keep up with security patches released by the virtualization-software vendor. Such software may have vulnerabilities that could allow malware to escape from the virtual system you infected and onto the physical host. Furthermore, don’t use the physical machine that’s hosting your virtualized lab for any other purpose.

Step 3: Install behavioral analysis tools:

     Before you’re ready to infect your laboratory system with the malware specimen, you need to install and activate the appropriate monitoring tools. Free utilities that will let you observe how Windows malware interacts with its environment include:

 * File system and registry monitoring: Process Monitor with ProcDOT offer a powerful way to observe how local processes read, write, or delete registry entries and files. These tools can help you understand how malware attempts to embed into the system upon infection.

           http://www.procdot.com/downloadprocdotbinaries.htm

  https://technet.microsoft.com/en-us/sysinternals/bb896645.aspx

* Process monitoring: Process Explorer and Process Hacker replace the
 built-in Windows Task Manager, helping you observe malicious processes, including
local network ports they may attempt to open.

                https://technet.microsoft.com/en-us/sysinternals/bb896653.aspx

                http://processhacker.sourceforge.net/

 * Network monitoring: Wireshark is a popular network sniffer, which can observe laboratory network traffic for malicious communication attempts, such as DNS resolution requests, bot traffic, or downloads.

       https://www.wireshark.org/

* Change detection: Regshot is a lightweight tool for comparing the system’s state before and after the infection, to highlight the key changes malware
  made to the file system and the registry.

  https://sourceforge.net/projects/regshot/

Behavioral monitoring tools can give you a sense for the key capabilities of malicious software. For further details about its characteristics, you may need to roll up your sleeves and perform some code analysis.

Step 4: Install code-analysis tools:

      Examining the code that comprises the specimen helps uncover characteristics that may be difficult to obtain through behavioral analysis. In the case of a malicious executable, you rarely will have the luxury of access to the source code from which it was created. Fortunately, the following free tools can help you reverse compiled Windows executables:

* Disassembler and debugger: OllyDbg and IDA Pro Freeware can parse compiled Windows executables and, acting as disassemblers, display their code as assembly instructions. These tools also have debugging capabilities, which allow you to execute the most interesting parts of the malicious program slowly and under highly controlled conditions, so you can better understand the purpose of the code.

https://www.hex-rays.com/products/ida/support/download_freeware.shtml

http://www.ollydbg.de/

* Memory dumper: Scylla and OllyDumpEx help obtain protected code located in the lab system’s memory and dump it to a file. This technique is particularly useful when analyzing packed executables, which are difficult to disassemble because they encode or encrypt their instructions, extracting them into RAM only during run-time.

             https://github.com/NtQuery/Scylla

             http://low-priority.appspot.com/ollydumpex/

Step 5: Utilize online analysis tools:

       To round off your malware-analysis toolkit, add to it some freely available online tools that may assist with the reverse engineering process. One category of such tools performs automated behavioral analysis of the executable you supply. These applications look similar at first glance, but use different technologies on the back end. Consider submitting your malware specimen to several of these sites; depending on the specimen, some sites will be more effective than others. Such tools include:

http://www.threatexpert.com/submit.aspx

http://anubis.iseclab.org/

https://malwr.com/

https://cuckoosandbox.org/

https://www.file-analyzer.net/

Virustotal.com

    Hybrid-analysis.com

You can see a longer list of free automated malware analysis services that can examine compiled Windows executables.

           https://zeltser.com/automated-malware-analysis/

 Another set of potentially useful online tools provides details about websites that are suspected of hosting malicious code. Some of these tools examine the sites you specify in real time; others provide historical information. Consider submitting a suspicious URL to several of these sites, because each may offer a slightly different perspective on the website in question:

    * Real-time threat assessment: WebInspector, and Wepawet

          http://app.webinspector.com/

http://wepawet.iseclab.org/

* Historical reputation data: URLVoid and MxToolbox

           http://www.urlvoid.com/

          http://mxtoolbox.com/blacklists.aspx

Step 6: Next Steps:

        With your initial toolkit assembled, start experimenting in the lab with malware
you come across on the web, in your e-mail box, on your systems, and so on. There are several
“cheat sheets” that can help you in this process, including:

Analyzing Malicious Software :
     https://zeltser.com/reverse-malware-cheat-sheet/

Analyzing Malicious Documents :
   https://zeltser.com/analyzing-malicious-documents/

 Os for Malware analysis : https://remnux.org/

Check Dyre malware Analysis for more details about Malware Analysis :
  https://ctechz.blogspot.in/2017/05/dyre-malware-analysis.html