Rewriting (proxying) a TCP stream in one line

Damn unix is great...

Step 1: In a shell, set up netcat to listen on TCP 5500:
nc -l -p 5500

Step 2: In another shell/on an intermediate machine..
nc -l -p 50000 -c 'sed -u -e s/billy/freddy/g | nc localhost 5500'

Step 3: In a third shell,
nc localhost 50000

..and start typing stuff, including the word "billy"

Ok so the third shell will talk to the first shell and replace "billy" with "freddy".
Note this is bidirectional - type in first shell and it goes back to third one (without rewrite)

How handy is that? VERY!

If you need a TCP stream (e.g. gnu debugger) intercepted and rewritten slightly.. that's yer man.

Thank you beardy unix gods from 1972!

Setting Tomcat JVM heap size as a percentage of total ram

Put at start of "bin/startup.sh"

export CATALINA_OPTS="-Xmx`cat /proc/meminfo | grep MemTotal | awk '{ print $2*0.75 } '`k"


..to use 75% of your total system memory for JVM.
Add to /bin/tomcat.sh or whatev's

SSL performance of Tomcat and Apache Commons HTTP

Using latest Tomcat 6 on Java 1.6 on Centos.

1. Tomcat JSSE (standard) SSL option is much slower than the Apache Portable Runtime (APR) libaries.

SSL performance (using CURL to upload large binaries via HTTPS to localhost on a multicore box) was getting
5.3MBytes/sec using Tomcat default SSL (100% Java on one CPU), and getting >30MBytes/sec using Tomcat with APR (basically OpenSSL) and lower CPU usage.

Conclusion: Well worth the hassle installing APR for incoming (and probably outgoing) e.g. 5x faster SSL traffic, lower less Java CPU.

2. Beware using standard libraries to upload to Rackspace Files and other HTTPS-accessed key:value stores. Using Apache Commons HTTP Uploader tops out at about 2mbytes/sec on one machine compared to 5mbytes/sec using CURL or WGET from command line. This was very repeatable.

3. Speed writing from Rackspace Cloud Server to Rackspace Files using CURL (i.e. running flat-out)
(See that the different CLoud Server options have different bandwidth clamps as well as significantly faster CPU)
Uploading a 176MB object to Rackspace Files, using CURL from command line to do a PUT
"512MB machine" : 5.02MBytes/sec (very reliably - this is clamped by network)
"1GB machine" : 7.65MBytes/sec (again network bound, more expensive server)
"16GB machine" : 15MBytes/sec
CPU in these cases (for Curl) is <10% typically.
In all cases running two threads (these are multicore machines) gets exactly half the performance, indicating networtj throttle.

HOWEVER if you don't use OpenSSL for your HTTPS....

Using Apache HTTP Uploader (and most likely any 'pure'-Java SSL options that are using JSSE) only gets
"512MB machine" : 2.1MBytes/sec (this is 100% CPU on that java thread)

Conclusion:
...creating outbound HTTPS connections using Java SSL gets CPU-bound really quickly - watch for this when using JSSE-based Java SSL clients.

Read facebook images in flash

var loaderContext:LoaderContext;
loaderContext= new LoaderContext();
loaderContext.securityDomain=SecurityDomain.currentDomain;
loaderContext.checkPolicyFile = true;

then pass that as the context parameter to load()

If you're using the excellent "BulkLoader" pass it as "context" param

Jetty + BlazeDS in NIO (async) mode

Ok so BlazeDS uses blocking IO normally, which makes it crap for when you're using long-polling clients because you run out of threads.

You can run it under Jetty instead, which supports continuations, which means that you can have thousands of (mostly idle) connections with only a few threads.

Basically follow Andrea blog with her patched code to run it in Jetty async mode...

http://blogs.webtide.com/athena/entry/asynchronous_blazeds_polling_with_jetty

..however it doesn't quite work with Jetty 7.1; if you apply this patch, it works!

http://jira.codehaus.org/browse/JETTY-1213

RSA-PSS signing with OpenSSL and Bouncycastle

Verified:
Bouncycastle RSA-PSS works with OpenSSL RSA-PSS (if you can find out how to do it ;-)) which also accept the real test vectors provided by RSA.

Well done those folks.


How:

Code to use:
Use the Bouncycastle signing example code, it's super simple.

OpenSSL: Use this

/*
 * An implementation of RSA PSS digital signature using OpenSSL
 *
 * Copyright (c) 2009 Mounir IDRASSI . All rights reserved.
 * Modified 2010 by Blitter Nasty to interoperate with Bouncycastle properly.
 *
 * This program is distributed in the hope that it will be useful, 
 * but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY 
 * or FITNESS FOR A PARTICULAR PURPOSE.
 *
 * Blitternasty's "With with Bouncycastle, dammit!" Mods are (from Mounir's code)...
 *  Use Sha1 not Sha256. 
 *  Use fixed Salt length of 20 when signing (not -2 = "maximum")
 *
 * With these changes the sigs generated are accepted by Bouncycastle Java library. Nice.
 * 
 */

#include <stdio.h>
#include <string.h>
#include <openssl/err.h>
#include <openssl/evp.h>
#include <openssl/rsa.h>
#include <openssl/rand.h>


int main(int argc, char** argv)
{
   RSA* pRsaKey = NULL;
   unsigned char pDigest[32];
   size_t uDigestLen = 32;
   const char* szMessage = "This is the string to be signed";
   EVP_MD_CTX md_ctx;
   unsigned char EM[128];
   unsigned char pSignature[128];
   unsigned char pDecrypted[128];
   int status = 0;

   /* openssl initialization */
 ERR_load_crypto_strings(); 
 OpenSSL_add_all_algorithms();

#ifdef _WIN32
 RAND_screen();      
#else
   RAND_poll();
#endif

   /* Generate an RSA key pair */
   pRsaKey = RSA_generate_key(1024, 0x010001, NULL, NULL);
   if (!pRsaKey)
   {
      printf("RSA_generate_key failed with error %s\n", ERR_error_string(ERR_get_error(), NULL));
      goto prog_end;
   }

   /* hash the message */
 EVP_MD_CTX_init(&md_ctx);
   EVP_DigestInit(&md_ctx, EVP_sha256());
 EVP_DigestUpdate(&md_ctx, (const void*) szMessage, strlen(szMessage));
 EVP_DigestFinal(&md_ctx, pDigest, &uDigestLen);
   EVP_MD_CTX_cleanup(&md_ctx);

   /* compute the PSS padded data */
   status = RSA_padding_add_PKCS1_PSS(pRsaKey, EM, pDigest, EVP_sha1(), 20 /* fixed salt length! (-2 didn't work for me)*/);
   if (!status)  
   {
      printf("RSA_padding_add_PKCS1_PSS failed with error %s\n", ERR_error_string(ERR_get_error(), NULL));
      goto prog_end;      
   }

   /* perform digital signature */
   status = RSA_private_encrypt(128, EM, pSignature, pRsaKey, RSA_NO_PADDING);
   if (status == -1)
   {
      printf("RSA_private_encrypt failed with error %s\n", ERR_error_string(ERR_get_error(), NULL));
      goto prog_end;      
   }



   // At this point you have a working RSA-PSS signature that works with Bouncycastle RSA-PSS validation code.
   // The sig data is 128 bytes always with this code.


   /* now we will verify the signature 
      Start by a RAW decrypt of the signature
   */
   status = RSA_public_decrypt(128, pSignature, pDecrypted, pRsaKey, RSA_NO_PADDING);
   if (status == -1)
   {
      printf("RSA_public_decrypt failed with error %s\n", ERR_error_string(ERR_get_error(), NULL));
      goto prog_end;      
   }

   /* verify the data */
   status = RSA_verify_PKCS1_PSS(pRsaKey, pDigest, EVP_sha1(), pDecrypted, -2 /* salt length recovered from signature*/);
   if (status == 1)
   {
      printf("Signature verification successfull!\n");
   }
   else
   {
      printf("RSA_verify_PKCS1_PSS failed with error %s\n", ERR_error_string(ERR_get_error(), NULL));
      goto prog_end;            
   }

prog_end:
   if (pRsaKey)
      RSA_free(pRsaKey);

   /* openssl cleanup */
   CRYPTO_cleanup_all_ex_data();
   RAND_cleanup();
 EVP_cleanup();
 ERR_free_strings();
   ERR_remove_state(0);

   return 0;
}

Using HTMLCleaner in Java to convert your pages to DOM and then back again, but finding the pages come up blank in a brower?

    private void fixScriptTagWierdness(TagNode document) {
        //name says it all - symptom is a blank page in Firefox and IE, works fine in Chrome. Silly browsers require that " and doesn't have (valid?) ..../> ending.
        //hence we stick some junk inside the element and it expands it and everything workie.
        TagNode scriptTag=document.findElementByName("script", true);
        if (scriptTag!=null)
        {
            scriptTag.addChild(new TagNode("junk"));
        }
    }

Laptop fan making grinding noises? Cheap and easy fix

I recently upgraded my laptop to use a Solid State Disk instead of a hard drive. This makes it very quick and very quiet, but also reduces the total count of moving parts to three;
a) the hinge for the display
b) the keyboard
c) the cooling fan
I don't close the display nor use the keyboard (have an external one) so there is only one moving part in the whole thing; the fan.

Last night my laptop started making horrible grinding noises, so I took it apart, blew out all the dust that had accumulated in the cooling section, and head straight for the only thing that could be making noise;

The fan in question is about 2" across;
http://www.notebook-doctor.com/popup_image.php?pID=2220300

..and from this website I find I can buy a new "thermal assembly" for .. $139. Nice.
This fan costs maybe 75 cents to make, and itself has only one single moving part; the spinning plastic fan blades, which are mounted on a metal shaft with a permanent magnet (there's a fixed set of electromagnets that make it spin). Ok so literally the only surface in the entire laptop that has any wear is where 5mm long metal shaft of the fan blades meets the fan housing, which sits in a minscule bearing; total size maybe 1mm square.

This, of course, was making the grinding noise; dust had got in there.

I cleaned it and put the tiniest drop of cooking oil on it, and now it's silent again. The whole machine should be good for another five years or 315360000000000000 CPU cycles until it needs another single droplet of oil.

If that worked for you, post a comment

Problem installing RIM COD modules on Blackberry 4.x and 5.0 using CodeModuleManager APIs - no permissions

There is a bug in the RIM Code Module Manager installation APIs that causes newly installed apps to have no permissions until the next reboot.

The suggested fix is to be working for a company that does around $100m a month of business with RIM; this worked wonderfully, and ensured my problem was fixed by developer support in no time.

Seriously? Ok the actual solution (found via above method) is to use some private RIM APIs for which you need special JAR goodness; contact dev support and ask them nicely about it.

Proper Symlinks in Vista+Win7 - who knew?

Well f'ing hell nobody told me that Windows had Got With The Program and introduced proper, real, honest-to-goodness Symlinks in NTFS nowadays.

use dos command: (console must be 'run as administrator', right-click that option when opening dos box or you get an error)

mklink

Works great; can symlink your shit all over the place, between drives, etc.


Yay!

I have to say that Win7 seems pretty good so far. Mind you, I had no great issues with Vista, apart from them removing 'telnet' ;-p

Note: mklink is somewhat braindead of course; you have to add \d if you want a link to a directory (why? can't it tell?), otherwise it simply creates you a link that.. doesn't do anything. Splendid work there ;-)

Error 1021 cannot create registry key when installing VMWare 7.0.1 on Win7

All sorts of stuff on the net about "set US locale" and "install with -L English" none of which worked for me.
I'd previously run VMWare off another disk without it being installed; it clearly created a registry entry which caused problems with subsequent install of full app.

Solution:
Use regedit to delete the key /HKLM/SOFTWARE/{Wow6432Node}/VMware,Inc.
You'll only have the Wow.. stuff if you're running on 64 bit.
Once this key is gone the installer worked (for me).

I did have it set in 'run as administrator' and 'run in vista compatibility mode' also, but I don't think that's what fixed it.