Monday 16 February 2015

Running debuggable tests & coverage on the same karma server

Karma is one of the more popular javascript testing frameworks. But so far, there has been no option to run debugging & coverage on the same server. This issue’s been sitting on github for some years.

The main issue is that karma uses istanbul under the hood for coverage, which does not support source maps. Furthermore, due to istanbul’s detailed branch coverage, debugging, even if it were possible, would be rather slow. This appears to introduce the need for running two karma instances, one configured for debugging, another for coverage.

But there is a much simpler way. What if we were to include both builds within the same page? This configuration is too complex for karma’s build system to support without writing additional plugins, but hey, why not use the popular nodejs build systems directly? Here’s a simplified setup of mine which does exactly this:

var gulp = require('gulp');
var concat = require('gulp-concat-util');
var istanbul = require('gulp-istanbul');
var watch = require('gulp-watch');
var karma = require('karma');
var path = require('path');

gulp.task('test', function(){
  karma.server.start({
    frameworks: ['qunit'],
    reporters: ['coverage', 'progress'],
    autoWatch: false,
    files: ['dist/**.js']
  });
  watch('**/*.js', {}, function(){
    gulp.src('**/*.js')
      .pipe(concat.header('if(window.location.href.endsWith("debug.html")){'))
      .pipe(concat.footer('}'))
      .pipe(gulp.dest('dist/test'));
    gulp.src('**/*.js')
      .pipe(istanbul())
      .pipe(istanbul.hookRequire())
      .pipe(concat.header('if(!window.location.href.endsWith("debug.html")){'))
      .pipe(concat.footer('}'))
      .pipe(gulp.dest('dist/cover'))
      .on('end', function(){
        // monkey-patch to get rid of stdout
        var http = require('http');
        var request = http.request;
        http.request = function(options, response){return request(options);};
        karma.runner.run({});
        http.request = request;
      });
  });
  open('http://localhost:9876/');
});

The concat magic determines which script to run based on the current url (debug.html is, quite simply, karma’s debug page), and run fires off the signal to run tests.

If you’ve been reading carefully, the monkey-patch may appear to be rather confusing, but that’s due to karma.runner.run outputting the test results to the console, which is what the server’s doing at the same time. We wouldn’t want to have two outputs writing over each other, would we? (bug’s filed here).

Happy debugging!

Tuesday 30 December 2014

Writing a JS sol parser to cheat idle farmer

Idle farmer was a game I discovered a few days ago which was one of the more enjoyable incremental games. Unfortunately, farming in the game is slow and tedious, and only gets worse over time. I decided to speed it up a little.

Scanning variables in cheat engine yields no results, for reasons I don’t know (and haven’t investigated), so I decided to look at the sol data. Converted to json, it looks something like this:

{
    "dataos": "gJE5706:321250LB<9Â ... "
}

Clearly, this is some form of encrypted data. I hooked the game up in the JPEXS decompiler, and after some digging, found these two classes related to save data handling:

DataSaver
DatosGuardados

DatosGuardados is the class which handles save data encryption/decryption, and DataSaver is the one which convert’s it into an actionscript format.

Something new here which isn’t usually seen in other flash games is it’s use of the haxe serialization/deserialization routines. So I took the lazy route to write a parser in js, the closest language to both haxe and actionscript which is the easiest to hack with.

There’s no existing sol parser for js, but sol is really just a thin container around amf3, and there already are some libraries for dealing with it, so this wasn’t so hard.

Next came the deserialization part, which involved copy pasting DataSaver and DatosGuardados and manually converting them to haxe (the language similarities meant that this was relatively easy work).

Then came the hard part. The decrypted data was not unserializable, nor human readable. There weren’t any avm bytecode debuggers to help either. How are we going to know what fucked up?

But luck is on our side, and the actionscript trace function comes to the rescue. We can basically print anything by inserting the following in the avm code:

findpropstrict Qname(PackageNamespace(""),"trace")
getlocal 1
callpropvoid Qname(PackageNamespace(""),"trace") 1

and enabling flash player tracing in mm.cfg.

After printing some loop variables and inspecting the code which clearly didn’t look right, the bug became apparent:

_loc14_ = 0;
while(_loc14_ < _loc6_) {
    _loc14_++;
    _loc15_ = _loc14_;
    _loc11_ = param1.charCodeAt(_loc13_ - _loc15_ - 1);
    _loc2_.b = _loc2_.b + String.fromCharCode(_loc11_ - _loc10_);
}

(param1 is the input string, looks like an off-by-one here)

The assignment translates to the following, can you spot the error?

getlocal 14
inclocal_i 14
convert_i
setlocal 15

getlocal pushes local 14 onto the stack, inclocal_i doesn’t modify the stack, so convert_i is still using the old value - _loc14_, not _loc14_ + 1.

The fix is simple:

_loc14_ = 0;
while(_loc14_ < _loc6_) {
    _loc15_ = _loc14_;
    _loc14_++;
    _loc11_ = param1.charCodeAt(_loc13_ - _loc15_ - 1);
    _loc2_.b = _loc2_.b + String.fromCharCode(_loc11_ - _loc10_);
}

Now we can finally deserialize, pretty print everything in json, make the exporter, and be done with it!

And if you feel like cheating today, git clone https://github.com/simonzack/idle_farmer_parser.

Oh, and if you’re wondering about the strange key names, they (and the game) are written in spanish.

Wednesday 24 December 2014

Disabling smooth scrolling on wordpad

I’ve always been annoyed by smooth scrolling, and if you’re reading this blog post, you’re probably annoyed too. Wordpad on windows has smooth scrolling on, and a quick browse through the settings turns up no option to disable this. Time to debug!

A quick browse through the modules list (windbg’s lm) shows up something interesting, msftedit.dll, which a quick googling shows that it allows applications to embed rich text text-boxes. This is the dll for the latest version of the rich text edit control, v4.1. Previous versions include riched20.dll and riched32.dll.

Since that is basically what wordpad is doing, could wordpad simply have embedded a rich text box in it’s window? Breaking on msftedit!RichEditWndProc shows that this is indeed the case, where scrolling the text box breaks on this routine, with WM_MOUSEWHEEL as the message parameter.

Rich edit boxes are not only used under wordpad, but all over the place. Examples include tortoisegit and windbg.

Digging a bit further, we see that the routine that starts the scrolling is the following:

0051ee3c 5ec19ed8 Msftedit!CDisplay::SmoothVScroll
0051ee60 5ec0d6de Msftedit!CMagellan::MagellanRollScroll+0x98
0051eea8 5ebb8f66 Msftedit!CTxtEdit::HandleMouseWheel+0xee
0051f028 5ebe38e5 Msftedit!CTxtEdit::TxSendMessage+0xe15
0051f918 76de62fa Msftedit!RichEditWndProc+0xa31

Fortunately for us, SmoothVScroll is open sourced in windows CE, which helps in analysis. But going through each of the methods starting from HandleMouseWheel, we see that there are a couple of places the call to SmoothVScroll can be skipped, depending on some flags. Unfortunately none give us the desired behaviour. Some flags control zooming, and others disable scrolling entirely.

The relevant header file defines SmoothVScroll ‘s as:

VOID    SmoothVScroll ( int direction, WORD cLines, int speedNum, int speedDenom, BOOL fMouseRoller );

We can see some speed arguments here. Patching speedNum to 1 and speedDenom to 50, to give a speed of 1/50 instead of the default 3/1, does give us instant scrolling behavour, But SmoothVScroll also has a kind of acceleration effect, where if we scroll longer using the mouse wheel the scrolling speed will accelerate. This can be rather annoying while editing text.

There is a way to skip the call to SmoothVScroll entirely, and without going into internal functions, by hooking RichEditWndProc and sending WM_VSCROLL when WM_MOUSEWHEEL is received. The former occurs when we press the up and down buttons on the scrollbar.

We can also scroll faster by sending it more than once.

I implemented the above hooks in rich edit scroll, which uses minhook, SetWindowsHookEx and friends. Run install.bat to run it on startup, and smooth scrolling will be gotten rid of for good.

Rich edit scroll allows configuring which applications to hook as well as scrolling speed. Hooking is opt-in due to exe protectors often checking whether LoadLibrary is hooked, which is sometimes necessary, in the case of windbg. That can cause them to stop working.

Wednesday 22 October 2014

Consolidating orphaned vmware snapshot vmdks

VMware often leaves behind orphaned vmdks after snapshots are deleted. There is no clear way to consolidate this into the main vmdk file from the gui. But by googling around, it appears that people are fixing this by cloning a virtual machine.

But there is a much better and less intrusive method. Internally, vmware clones a virtual machine by using vmware-vdiskmanager.exe, which can be found in the vmware installation directory. It appears that the following command will consolidate the snapshot in the vmdk to a different one (beware, it does not accept paths containing spaces, and will give some spurious error messages).

"C:\Program Files (x86)\VMware\VMware Workstation\vmware-vdiskmanager.exe" -r main-000002.vmdk -t 0 C:\test.vmdk

The story should’ve ended here, but vmware-vdiskmanager.exe gave me a spurious error message:

Failed to convert disk: One of the parameters supplied is invalid (0x100003e80).

Googling this gave me dozens of dozens of problems associated with this generic error message. But vmware-vdiskmanager.exe does in fact have logs, located in the %temp%\vmware-%username%. Scrolling through these logs, vdiskmanager.log. Scrolling through this, some less generic but not any less confusing error message showed up:

DISKLIB-LIB   : DB: incorrect set operation 'consolidateDestFileName' = '(null)'.
DISKLIB-LIB   : Failed to clone : One of the parameters supplied is invalid (1).

DISKLIB-LIB   : DB: incorrect set operation 'resumeConsolidateSector' = '(null)'.
DISKLIB-LIB   : Failed to clone : One of the parameters supplied is invalid (1).

The fields referenced by the error isn’t documented anywhere at all. However, by looking at the snapshot vmdk’s using a hex editor, it appears that these are internal settings within the vmdk files. consolidateDestFileName should point to the main vmdk file, but was missing in the snapshot vmdk file vmware-vdiskmanager.exe choked on. Copying this into the snapshot vmdk file seemed easy enough, but I had no idea what resumeConsolidateSector meant.

Fortunately, it appears that these fields can be restored by creating a new snapshot in vmdk and deleting it. vmware-vdiskmanager.exe finally started working correctly, and I managed to consolidate my snapshots.

Monday 13 October 2014

Firefight Trainer

From the comments looks like a lot of you are still interested in firefight. Old games are still the best!

I did some work over the years and tidied some of it up last few weeks ago to create a cheat engine trainer.

Download the latest version of cheat engine, and the trainer, attach to the process, and fire it up.

Enjoy!

Sunday 21 August 2011

Patching utorrent slow ipfilter loading

When utorrent's ipfilter.dat, a file which filters out malicious ips in peer swarms, grows large, utorrent hangs at startup
I noticed this when I tried putting some new lists into the filter

A little reversing uncovers the issue

The ipfilter.dat ip range format looks like this (for ipv4)
1.2.3.4-5.6.7.8,0,test

This includes a start, an end, a priority level, and an optional description

The problem happens after utorrent parses ipfilter.dat, when it attempts to optimize it
It does so by trying to merge all the ip filter ranges together

the function is layed out like this:
sort(ipRanges)
for ipRange in ipRanges:
    if ipRange overlaps with the previous range:
        copy ipRanges[i:] to ipRanges[i-1:]

when there are many overlapping ranges, the copy operation (a memcpy) becomes very slow

obviously, when there are no overlapping ranges, the slow down does not occur

there are many ways to rewrite this, as long as only 1 range is copied every compare, the function will be fast

Patching this makes utorrent start-up fast again

I'm not sure if utorrent 3 has fixed this yet, but for those who are still using v2, here's a patch:
http://www.mediafire.com/?tto3p44mwvn5tfx

Tuesday 21 June 2011

eset crackme 2011 solution

I've promised to post a solution, so here it is

the crackme does some simple checking first

len(name)>=5
len(key)>=0xA

then it loads an elf file from resources, which doesn't have anything linux-specific except for the file format, so the parts of it used runs fine on windows

it calls 2 routines from it

the first routine hashes the username, identified by the genius cryptographist dcoder as cubehash

this hash is then fed into a big macro laden routine

it looks big, however when you look at the individual parts it's not that complicated

inside the big loop:
the first macros reverses the bits of a 64 bit var
the next few calls another macro 32 times, we can simplify it to this

esi=bitset(local_134,63)?5043443f1755df4d:0
ecx=bitset(local_134,62)?5043443f1755df4d:0
ecx:edi^((esi:ebx^(local_138:local_134<<1))<<1) simplifying further, it actually calls one macro twice n=((n>>63)?0x5043443f1755df4d:0)^(n<<1), where n is a 64 bit number

the last few macros reverse the bits of n again

this looks very much like crc, in fact, it is a crc64 with a starting value of:
0x3537000000003735
and a polynomial of:
0xb2fbaae8fc22c20a (this is the bit reversed 0x5043443f1755df4d)
the final value is xored with:
0x6b3e997a6008e054

a constant string and our key is fed into this routine, then compared with the hash

to reverse this, we simply reverse the crc

most of eset's crackmes so far needs a certain crc to be reversed, so you better get used to it if you do more :)