Rewriting (proxying) a TCP stream in one line

Damn unix is great...

Step 1: In a shell, set up netcat to listen on TCP 5500:
nc -l -p 5500

Step 2: In another shell/on an intermediate machine..
nc -l -p 50000 -c 'sed -u -e s/billy/freddy/g | nc localhost 5500'

Step 3: In a third shell,
nc localhost 50000

..and start typing stuff, including the word "billy"

Ok so the third shell will talk to the first shell and replace "billy" with "freddy".
Note this is bidirectional - type in first shell and it goes back to third one (without rewrite)

How handy is that? VERY!

If you need a TCP stream (e.g. gnu debugger) intercepted and rewritten slightly.. that's yer man.

Thank you beardy unix gods from 1972!

Setting Tomcat JVM heap size as a percentage of total ram

Put at start of "bin/startup.sh"

export CATALINA_OPTS="-Xmx`cat /proc/meminfo | grep MemTotal | awk '{ print $2*0.75 } '`k"


..to use 75% of your total system memory for JVM.
Add to /bin/tomcat.sh or whatev's

SSL performance of Tomcat and Apache Commons HTTP

Using latest Tomcat 6 on Java 1.6 on Centos.

1. Tomcat JSSE (standard) SSL option is much slower than the Apache Portable Runtime (APR) libaries.

SSL performance (using CURL to upload large binaries via HTTPS to localhost on a multicore box) was getting
5.3MBytes/sec using Tomcat default SSL (100% Java on one CPU), and getting >30MBytes/sec using Tomcat with APR (basically OpenSSL) and lower CPU usage.

Conclusion: Well worth the hassle installing APR for incoming (and probably outgoing) e.g. 5x faster SSL traffic, lower less Java CPU.

2. Beware using standard libraries to upload to Rackspace Files and other HTTPS-accessed key:value stores. Using Apache Commons HTTP Uploader tops out at about 2mbytes/sec on one machine compared to 5mbytes/sec using CURL or WGET from command line. This was very repeatable.

3. Speed writing from Rackspace Cloud Server to Rackspace Files using CURL (i.e. running flat-out)
(See that the different CLoud Server options have different bandwidth clamps as well as significantly faster CPU)
Uploading a 176MB object to Rackspace Files, using CURL from command line to do a PUT
"512MB machine" : 5.02MBytes/sec (very reliably - this is clamped by network)
"1GB machine" : 7.65MBytes/sec (again network bound, more expensive server)
"16GB machine" : 15MBytes/sec
CPU in these cases (for Curl) is <10% typically.
In all cases running two threads (these are multicore machines) gets exactly half the performance, indicating networtj throttle.

HOWEVER if you don't use OpenSSL for your HTTPS....

Using Apache HTTP Uploader (and most likely any 'pure'-Java SSL options that are using JSSE) only gets
"512MB machine" : 2.1MBytes/sec (this is 100% CPU on that java thread)

Conclusion:
...creating outbound HTTPS connections using Java SSL gets CPU-bound really quickly - watch for this when using JSSE-based Java SSL clients.

Read facebook images in flash

var loaderContext:LoaderContext;
loaderContext= new LoaderContext();
loaderContext.securityDomain=SecurityDomain.currentDomain;
loaderContext.checkPolicyFile = true;

then pass that as the context parameter to load()

If you're using the excellent "BulkLoader" pass it as "context" param

Jetty + BlazeDS in NIO (async) mode

Ok so BlazeDS uses blocking IO normally, which makes it crap for when you're using long-polling clients because you run out of threads.

You can run it under Jetty instead, which supports continuations, which means that you can have thousands of (mostly idle) connections with only a few threads.

Basically follow Andrea blog with her patched code to run it in Jetty async mode...

http://blogs.webtide.com/athena/entry/asynchronous_blazeds_polling_with_jetty

..however it doesn't quite work with Jetty 7.1; if you apply this patch, it works!

http://jira.codehaus.org/browse/JETTY-1213

RSA-PSS signing with OpenSSL and Bouncycastle

Verified:
Bouncycastle RSA-PSS works with OpenSSL RSA-PSS (if you can find out how to do it ;-)) which also accept the real test vectors provided by RSA.

Well done those folks.


How:

Code to use:
Use the Bouncycastle signing example code, it's super simple.

OpenSSL: Use this

/*
 * An implementation of RSA PSS digital signature using OpenSSL
 *
 * Copyright (c) 2009 Mounir IDRASSI . All rights reserved.
 * Modified 2010 by Blitter Nasty to interoperate with Bouncycastle properly.
 *
 * This program is distributed in the hope that it will be useful, 
 * but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY 
 * or FITNESS FOR A PARTICULAR PURPOSE.
 *
 * Blitternasty's "With with Bouncycastle, dammit!" Mods are (from Mounir's code)...
 *  Use Sha1 not Sha256. 
 *  Use fixed Salt length of 20 when signing (not -2 = "maximum")
 *
 * With these changes the sigs generated are accepted by Bouncycastle Java library. Nice.
 * 
 */

#include <stdio.h>
#include <string.h>
#include <openssl/err.h>
#include <openssl/evp.h>
#include <openssl/rsa.h>
#include <openssl/rand.h>


int main(int argc, char** argv)
{
   RSA* pRsaKey = NULL;
   unsigned char pDigest[32];
   size_t uDigestLen = 32;
   const char* szMessage = "This is the string to be signed";
   EVP_MD_CTX md_ctx;
   unsigned char EM[128];
   unsigned char pSignature[128];
   unsigned char pDecrypted[128];
   int status = 0;

   /* openssl initialization */
 ERR_load_crypto_strings(); 
 OpenSSL_add_all_algorithms();

#ifdef _WIN32
 RAND_screen();      
#else
   RAND_poll();
#endif

   /* Generate an RSA key pair */
   pRsaKey = RSA_generate_key(1024, 0x010001, NULL, NULL);
   if (!pRsaKey)
   {
      printf("RSA_generate_key failed with error %s\n", ERR_error_string(ERR_get_error(), NULL));
      goto prog_end;
   }

   /* hash the message */
 EVP_MD_CTX_init(&md_ctx);
   EVP_DigestInit(&md_ctx, EVP_sha256());
 EVP_DigestUpdate(&md_ctx, (const void*) szMessage, strlen(szMessage));
 EVP_DigestFinal(&md_ctx, pDigest, &uDigestLen);
   EVP_MD_CTX_cleanup(&md_ctx);

   /* compute the PSS padded data */
   status = RSA_padding_add_PKCS1_PSS(pRsaKey, EM, pDigest, EVP_sha1(), 20 /* fixed salt length! (-2 didn't work for me)*/);
   if (!status)  
   {
      printf("RSA_padding_add_PKCS1_PSS failed with error %s\n", ERR_error_string(ERR_get_error(), NULL));
      goto prog_end;      
   }

   /* perform digital signature */
   status = RSA_private_encrypt(128, EM, pSignature, pRsaKey, RSA_NO_PADDING);
   if (status == -1)
   {
      printf("RSA_private_encrypt failed with error %s\n", ERR_error_string(ERR_get_error(), NULL));
      goto prog_end;      
   }



   // At this point you have a working RSA-PSS signature that works with Bouncycastle RSA-PSS validation code.
   // The sig data is 128 bytes always with this code.


   /* now we will verify the signature 
      Start by a RAW decrypt of the signature
   */
   status = RSA_public_decrypt(128, pSignature, pDecrypted, pRsaKey, RSA_NO_PADDING);
   if (status == -1)
   {
      printf("RSA_public_decrypt failed with error %s\n", ERR_error_string(ERR_get_error(), NULL));
      goto prog_end;      
   }

   /* verify the data */
   status = RSA_verify_PKCS1_PSS(pRsaKey, pDigest, EVP_sha1(), pDecrypted, -2 /* salt length recovered from signature*/);
   if (status == 1)
   {
      printf("Signature verification successfull!\n");
   }
   else
   {
      printf("RSA_verify_PKCS1_PSS failed with error %s\n", ERR_error_string(ERR_get_error(), NULL));
      goto prog_end;            
   }

prog_end:
   if (pRsaKey)
      RSA_free(pRsaKey);

   /* openssl cleanup */
   CRYPTO_cleanup_all_ex_data();
   RAND_cleanup();
 EVP_cleanup();
 ERR_free_strings();
   ERR_remove_state(0);

   return 0;
}

Using HTMLCleaner in Java to convert your pages to DOM and then back again, but finding the pages come up blank in a brower?

    private void fixScriptTagWierdness(TagNode document) {
        //name says it all - symptom is a blank page in Firefox and IE, works fine in Chrome. Silly browsers require that " and doesn't have (valid?) ..../> ending.
        //hence we stick some junk inside the element and it expands it and everything workie.
        TagNode scriptTag=document.findElementByName("script", true);
        if (scriptTag!=null)
        {
            scriptTag.addChild(new TagNode("junk"));
        }
    }