Decrypting GPG Files On Windows Using Gpg4win

Using encryption is essential when transferring sensitive files from person to person.  Below are the steps to install Gpg4win and use it to decrypt gpg files on a Windows computer.
 

Installing Gpg4win

We want to only use the Kleopatra program from the Gpg4win bundle. There are several other cool plugins for Outlook et al. but for this we don't need all that.

1. Go to this site http://www.gpg4win.org/download.html
2. Click on the Full installer link
3. Download the installation file
4. When the download is complete open it up
5. Follow the install wizard but when you get to this page make sure all checks are unchecked except for Kleopatra:

 

Complete the installation and don't worry about reading the README file at the end.

Decrypt PGP File

1. Open the Kleopatra program

2. Go to the File menu and select Decrypt/Verify Files...

3. Find and select the file you want to decrypt

4. You will see a decryption settings page that looks like this:

5. For normal decryption (with symmetric keys) all you have to do is set the Output folder to tell Kleopatra where the decrypted file(s) should go.  Here we are decrypting an encrypted pdf file called test.pdf.gpg into the output folder E:/

6. When ready click on the Decrypt/Verify button

7. You will be prompted for the password:

8. After you enter the password click OK

9. If successful, you should see this:

10. Now, if you look in the output folder previously selected, you will see a file with the ".gpg" removed.  This is the decrypted file. So, for example, if you were decrypting test.pdf.gpg you will find test.pdf in the output directory.

Final Thoughts

This method is not the most secure way of transferring files but the setup is minimal when your recipient may not want to go through the extra steps needed to use public key encryption.  Public key encryption is usually more secure than symmetric key encryption.  Gpg4win is very well made and has many bells and whistles.  It makes the use of public key encryption much easier.  So take a look at how to set up and use public key encryption in their manual.

Below I have listed a series of videos which describe:

1. Installing Gpg4win
2. Generating public/private keys
3. Encrypting/Decrypting files using public key encryption

To help share this information I have also publish this information here:
http://bit.ly/14cxURQ

 

 

Pivot Columns Values into a Set #PostgreSQL

Depending on the project you are working on, you may have to generate a set which contains every value found (across several columns) into a set consisting of a single column.

For example you have 3 columns A, B, C:

a1, b1, c1

a2, b2, c2

a3, b3, b3

and you need:

a1

a2

a3

b1

b2

b3

c1

c2

c3

To achieve this you can use the array function called unnest:

select unnest( array[A, B, C] )

from foo

This takes an array of tuples created by array[a,b,c] which looks like this:

{ {a1, b1, c1}, {a2, b2, c2}, {a3, b3, b3} }

and returns a set that contains every value in the array of column values.

Some caveats to consider:

1. The types of the columns must be equivalent or they must be cast
2. The unnest function is available in PostgreSQL 8.4 or greater

Filter the number of databases in pgAdmin

Quick post about reducing the amount of scrolling you have to do in pgAdmin. If you work on a PostgreSQL instance with 124 databases like I do, it helps to be able to filter the list of databases so only the databases you work on show up. This way you don't have to scroll and scan for the DB you want.
To do this:

1. Right click on the connection to the instance you want to change
2. Click on Properties
3. In the "DB restriction" text box add the name of each database you work on single quoted and comma delimited. For example: 'database1','database21','database101'
4. Restart pgAdmin and you are all set. Now you will only see the databases you work with.

Running Amazon's Kindle app on Linux

I was trying to add a comment to this article which goes through the steps needed to get the amazon kindle application running on Linux. The instructions do not work as stated because many people running Ubuntu 10.10 have wine 1.2 which does not work with Amazon's application.

Here is the original post, which is the first post that comes up with you Google it.

My comment was flagged as spam, so I wanted to make sure people know how to get it working. Here is my comment:

This is working for wine 1.3.15 on Ubuntu 10.10. It does not work with the stock version of Wine that is in the default Ubuntu 10.10 repo.
To get the latest version of Wine, you have to add the WineQH repository by following these instructions: http://www.winehq.org/download/deb

After that, uninstall any wine 1.2 packages and install the package called "wine1.3"
I used the Synaptic Package Manager which you can find under
System->Administration

Alternatively you can run the following on the command line after you have uninstalled any wine 1.2 packages you may have:
sudo add-apt-repository ppa:ubuntu-wine/ppa
sudo apt-get update
sudo apt-get install wine1.3

To find what is currently working go here

Hope this helps and happy reading!

@Genspire

Mimicking Hudson when you must

Just a quick blog about a situation I found myself in because my current employer refuses to use Hudon for CI testing of PHP projects.

I was looking for a way to use phpunit to run tests and create a website that nicely displays the results. I thought it would be simple to create an ant script which would run the tests using the --log-junit feature to create an xml document that the junitreport ant task can use to produce a quick and dirty website that will display the test stats. It turns out that the xml that phpunit emits is not well formed junit xml, so the ant task would not parse it.

I could have used an xslt file and linux's xsltproc to translate the phpunit xml into what junitreport needs but I don't think this is robust solution into the future.

So instead I found this website:
http://clockwerx.blogspot.com/search/label/prototype.js

Here Daniel uses the phpunit json logging feature (--log-json) to emit json which is then used in a simple php file to display the test results. It also allows you to filter based on the status of each test.

I took his code and converted it to use jQuery (pulled down using google api so you don't need to download it yourself) and moved the filter div to the top so it doesn't cover test results. Also, I removed the links to raw output, agile, and recent pages.

Since we run the tests nightly (or when there is an update), we email a unique link to our developers. So the page is expecting a test (get) parameter which is used to locate the correct json file to display.

The url sent to other developers will look like this:
http://localhost/results.php?test=2010-11-16_1219

You will have to find icons and place them in an 'images' directory located in the same directory as this file.

Here it is.. hope it helps you when your company decides not to use a proven tool:

<?php

$file = dirname(__FILE__) . '/' . $_GET["test"]. '.js';
if (!file_exists($file)) {
die("No unit tests have been run?<br/>Nothing found in " . $file);
}
$json = file_get_contents($file);

//Ugh
$json = '[' . str_replace("}{", "},{", $json) . ']';

?>
<html>
<head>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.4.4/jquery.min.js"></script>
</head>
<body>
<style>
table {
border: 1px solid rgb(240, 240, 240);
}

.error {
border: 1px solid orange;
}

.fail {
border: 1px solid red;
}

.pass {
border: 1px solid green;
}

.cell {
width: 200px !important;
text-align: center;
padding: 0.5em;
}
.cell img {
display: block;
margin: auto;
}

h4 {
font-weight: normal !important;
font-style: italic;
}
</style>
<script type="text/javascript">
var total_passes = 0;
var total_failures = 0;
var total_errors = 0;
var total_skipped = 0;
var total_incomplete= 0;

function renderSuiteStart(item) {
var h2 = document.createElement("H2");
h2.appendChild(document.createTextNode(item.suite));

var p = document.createElement("p");
p.appendChild(document.createTextNode(item.tests + " tests in suite"));

var div = document.createElement("div");
var table = document.createElement("table");

div.id = item.suite;
table.id = item.suite + "_table";
table.className = '';

div.appendChild(h2);
div.appendChild(p);
div.appendChild(table);

$('#main').append(div);
}

function renderTest(item) {
var h3 = document.createElement("H3");
var h4 = document.createElement("H4");
h3.appendChild(document.createTextNode(item.test));

var p = document.createElement("p");
p.appendChild(document.createTextNode(item.time + " tests in suite"));

var tr = document.createElement("TR");
var th = document.createElement("TH");
var message = document.createElement("TD");
var detail = document.createElement("TD");
var time = document.createElement("TD");
var img = document.createElement("IMG");
var backtrace = document.createElement("OL");


time.appendChild(document.createTextNode(item.time));


img.alt = item.status;
img.width="30";
img.height="30";
message.className = item.status + " cell";

switch (item.status) {
case 'error':
if (item.message == "Skipped Test") {
message.className = "skipped cell";
total_skipped++;
} else if (item.message == "Incomplete Test") {
message.className = "incomplete cell";
total_incomplete++;
} else {
total_errors++;
}
img.src = 'images/important.png';
break;
case 'fail':
total_failures++;
img.src = 'images/cross.png';
break;
default:
total_passes++;
img.src = 'images/check.png';
break;
}

$.each(item.trace,
function(i, t) {
var li = document.createElement("LI");
var text = t.file + "(line " + t.line + "): " + t.class + t.type + t.function + "()";
li.appendChild(document.createTextNode(text));

backtrace.appendChild(li);
}
);

h4.appendChild(document.createTextNode(item.test));
detail.appendChild(h4);
detail.appendChild(backtrace);

message.appendChild(img);
message.appendChild(document.createTextNode(item.message));
message.style.width = "300px";

tr.appendChild(message);
tr.appendChild(detail);


$("#"+ item.suite + "_table").append(tr);
}

function renderResults() {
var data = <?php print $json; ?>;

$.each(data, function(i,item) {

switch (item.event) {
case 'suiteStart':
renderSuiteStart(item);
break;
case 'test':
renderTest(item);
}
});
renderTOC();
}

function renderTOC() {
var menu = document.createElement('p');
var checkbox_pass = document.createElement('input');
var checkbox_fail = document.createElement('input');
var checkbox_error = document.createElement('input');
var checkbox_skipped = document.createElement('input');
var checkbox_incomplete = document.createElement('input');

var label_pass = document.createElement('label');
var label_fail = document.createElement('label');
var label_error = document.createElement('label');
var label_skipped = document.createElement('label');
var label_incomplete = document.createElement('label');
var status_image;

//menu.style.position = 'fixed';
menu.style.bottom = 0;
menu.style.right = "1.5em";
//menu.style.backgroundColor = 'rgb(240, 240, 240)';
menu.style.padding = "1em;";
menu.style.fontWeight = 'bolder';

checkbox_pass.type = "checkbox";
checkbox_fail.type = "checkbox";
checkbox_error.type = "checkbox";
checkbox_skipped.type = "checkbox";
checkbox_incomplete.type = "checkbox";

checkbox_pass.checked = "checked";
checkbox_fail.checked = "checked";
checkbox_error.checked = "checked";
checkbox_skipped.checked = "checked";
checkbox_incomplete.checked = "checked";


checkbox_pass.onclick=function(){toggle("pass")};
checkbox_fail.onclick=function(){toggle("fail")};
checkbox_error.onclick=function(){toggle("error")};
checkbox_skipped.onclick=function(){toggle("skipped")};
checkbox_incomplete.onclick=function(){toggle("incomplete")};

label_pass.appendChild(checkbox_pass);
label_fail.appendChild(checkbox_fail);
label_error.appendChild(checkbox_error);
label_skipped.appendChild(checkbox_skipped);
label_incomplete.appendChild(checkbox_incomplete);

label_pass.appendChild(document.createTextNode("Passed (" + total_passes + ")"));
label_fail.appendChild(document.createTextNode("Failed (" + total_failures + ")"));
label_error.appendChild(document.createTextNode("Error (" + total_errors + ")"));
label_skipped.appendChild(document.createTextNode("Skipped (" + total_skipped + ")"));
label_incomplete.appendChild(document.createTextNode("Incomplete (" + total_incomplete + ")"));

label_pass.style.color = "green";
label_fail.style.color = "red";
label_error.style.color = "#FF6600";
label_skipped.style.color = "blue";
label_incomplete.style.color = "gray";

status_image = document.createElement("IMG");
status_image.style.display = "block";
status_image.style.margin = "auto";
status_image.style.paddingright = "10px";
status_image.width="50";
status_image.height="50";
status_image.align="left";
if (total_failures > 0) {
status_image.src = "images/fail.gif";
status_image.title= "Some tests failed. Fix them and get a happy face";
} else {
status_image.src = "images/pass.jpg";
status_image.title= "All test passed!!! Give yourself a pat on the back, YOU ROCK!!";
}
menu.appendChild(status_image);

menu.appendChild(document.createElement("br"));
menu.appendChild(label_pass);
menu.appendChild(label_fail);
menu.appendChild(label_error);
menu.appendChild(label_skipped);
menu.appendChild(label_incomplete);

checkbox_pass.onclick();
checkbox_pass.checked = "";

checkbox_skipped.onclick();
checkbox_skipped.checked = "";

checkbox_incomplete.onclick();
checkbox_incomplete.checked = "";

$("#toc").append(menu);
}

function toggle(type) {

$("." + type).each(function(i, td) {
if (td.parentNode.style.display != 'none') {
td.parentNode.style.display = 'none';
} else {
td.parentNode.style.display = 'table-row';
}
});
}

$(document).ready(renderResults);
</script>
<div id="main" class="contentbox">
<h1>Unit tests</h1>
<p><!--<a href="results.txt">Raw results</a> | <a href="docs.html">Agile Documentation</a> | <a href="recent.php">Recent changes</a> | -->
Generated: <?php print date("F j, Y, g:i a", filemtime($file)); ?> </p>
<div id="toc"/>
</div>
</body>
</html>

I am looking for book ideas to refresh my machine learning knowledge. I have some but I wanted to see what the top universities are using these days.

Here are a couple of classes (Stanford has online courses):

Carnegie Mellon’s class:

• Textbook: Pattern Recognition and Machine Learning , Chris Bishop.
• Secondary textbook: The Elements of Statistical Learning: Data Mining, Inference, and Prediction Trevor Hastie, Robert Tibshirani, Jerome Friedman. 2nd edition.
• Optional textbook: Machine Learning , Tom Mitchell.
• Optional textbook: Information Theory, Inference, and Learning Algorithms , David Mackay.

MIT’s Class:

• Cowell et al., "Probabilistic networks and expert systems", Springer-Verlag, 1999.
• Bishop, "Neural Networks for Pattern Recognition", 1995
• Duda, Hart, Stork, "Pattern Classification", 2000
• Hastie, Tibshirani and Friedman, "Elements of Statistical Learning: Data Mining, Inference and Prediction", 2001
• MacKay, "Information Theory, Inference, and Learning Algorithms", 2003. Available on-line here

• Mitchell, "Machine Learning", 1997.
• Cover and Thomas, "Elements of Information Theory", Wiley & Sons, 1991

Stanford's Class:
Very Interesting Video Course: http://academicearth.org/courses/machine-learning

There is no required text for this course. Notes will be posted periodically on the course web site. The following books are recommended as optional reading:

• Christopher Bishop, Pattern Recognition and Machine Learning. Springer, 2006.
• Richard Duda, Peter Hart and David Stork, Pattern Classification, 2nd ed. John Wiley & Sons, 2001.
• Tom Mitchell, Machine Learning. McGraw-Hill, 1997.
• Richard Sutton and Andrew Barto, Reinforcement Learning: An introduction. MIT Press, 1998

Course handouts and other materials can be downloaded from http://www.stanford.edu/class/cs229/materials.html

Let me know if you have any opinions about these books or any I missed.

Debugging PHP with Eclipse on Ubuntu

I have followed the instructions at the site below but there are many differences I found when I was setting up PHP debugging using XDebug with Eclipse on Ubuntu 10.04. So I wanted to take you through the steps that worked for me.

Older/incomplete instructions are found here:
http://www.phpeclipse.com/wiki/Howto/XDebugAndPHPEclipse


Program installation
First you will have to install the following (we use postgres and smarty so that may not be required for you):

eclipse, php5, php5-pgsql, apache2, smarty, zendframework, php5-xdebug

you can run this:

sudo apt-get install eclipse php5 php5-pgsql apache2 smarty zendframework php5-xdebug

In eclipse add a new update site in Help->Install New Software
Click the Add button and enter this site url:
http://update.phpeclipse.net/update/stable/1.2.x

Select the root check box to install all components of the PHPEclipse plugin

XDebug set up

Edit xdebug.ini:

sudo nano /etc/php5/apache2/conf.d/xdebug.ini

find a line that looks like this (or you may need to add one):
zend_extension=/usr/lib/php5/20090626+lfs/xdebug.so

Add the following lines for a local debug session:

xdebug.remote_enable=On
xdebug.remote_autostart=On
xdebug.remote_handler=dbgp
xdebug.remote_host=127.0.0.1
xdebug.remote_port=9000
xdebug.remote_mode=req

Apache Configuration

Edit apache2.conf settings:

sudo nano /etc/apache2/apache2.conf

Add an alias and directory for your project (this is only for testing):

Alias /project_name "/home/your user/workspace/project directory"
ServerName test.net
< Virtualhost *:80>
     DocumentRoot /home/your user/workspace
     ErrorLog /error.log
     CustomLog /access.log common
</ Virtualhost>

If you do not have ErrorLog in here, the error long is located:
/var/log/apache2/error.log
This is defined in another section in the apache2.conf file as well.

Restart Apache:

sudo /etc/init.d/apache2 restart

Set up Eclipse remote debug session:

Click on this link and follow the instructions in section XDebug Debug Profile

Set a break point by opening the file with the Eclipse php editor and right clicking in the left margin next to line you want to break on. Then select "Toggle XDebug Breakpoint"

To start the debug session go to:
http://localhost/your_prj_sub_directory/?XDEBUG_SESSION_START=ide_id_string

Navigate to the place in your site that would execute the code with the breakpoint and start debugging.

The local variables can take some time to show up after you go to that tab. And if you run into trouble (null pointer exception) with the custom expressions (i.e. watch expressions), then remove them. The PHPEclipse plugin does not seem to be very robust in the way it handles any exceptions your php expression may throw while you are stepping through.

Some times Eclipse (or the PHP plugin) gets a little flaky.
When this happens try:

1. refresh the page


2. "hard refresh" (ctr-F5)


3. restarting the PHP XDebug Remote Script


4. all else fails you will have to restart eclipse

For more configuration options here are some important file locations for Ubuntu 10.04:
php.ini: /etc/php5/apache2/php.ini
httpd.conf: /etc/apache2/httpd.conf

If there is anything that I missed please let me know and I will update these instructions.

Hope this help.