How I automated taking a screenshot of every single Pokémon in Pokémon GO

Hello guys, hows it going, huh?

Today I am going to share you an extremely wild story about how I automated taking a screenshot of every single Pokémon in game!

The journey starts with a problem of not having every Pokémon in game. I mean.. Who does?

After all.. I am not going to ‘catch ’em all’, now am I? No! I am a programmer, not a Pokémon trainer..

First step was of course to get renders of all Pokémons.. Shouldn’t be too hard, right? Just manipulate, or more precisely.. Man in the middle the connection between my phone and the server, fake that the server sent every Pokémon in game and.. there we go, right?

After quick research I found this implementation in GitHub called Pokemon Go MITM Proxy by a nickname rastapasta, so I tried to setup it in Windows environment.. No luck.. obviously.. Its also documented in the repo that its hard to compile..

After recovering from the disappointment I cloned the repo on my Linux server and ran it there, installation went smoothly and I was ready to go!

Unfortunately there’s still need for root or removal of certificate pinning, but gladly from my last post I had a patch ready to go! So no worries! I installed the certificate as documented from the server and ran an example script that would replace all my pokémons, and I was good to go!

pokemon-list

When I ran Pokémon Go after this, I ended up getting a list like  this, however it wasn’t enough.. The example would only replace the Pokémons I had with others, but what I really needed is it to clear my Pokémons and add all Pokémons in increasing ids..

So its time to edit the script.. So I opened example.replacePokemons.coffee but coffeescript wasn’t too familiar with me.. Nor was the structure of packets, so first of all I had to lookup coffeescripts syntax before anything. It wasn’t too bad knowing Javascript, but required some effort to get through, but I found also this js2coffee compiler online I could use when stuck.

So the smallest modification I could get through with is remove all the code touching the pokemons, and craft a piece of code that replaces whole inventory with inventory cotaining an item with PlayerStats and all available Pokémons.

As funny as it sounds, PlayerStast too are passed as an InventoryItem.. Smart..

The minimal implementation I was left with is as follows:

PokemonGoMITM = require './lib/pokemon-go-mitm'
fs = require 'fs'

server = new PokemonGoMITM port: 8081
	.addResponseHandler "GetInventory", (data) ->
		if data.inventory_delta
			data.inventory_delta.inventory_items = [
				{
					inventory_item_data: {
						player_stats: {
							level: 99,
							experience: 999999,
							prev_level_xp: 1,
							next_level_xp: 1,
							km_walked: 0,
							pokemons_encountered: 2,
							unique_pokedex_entries: 1,
							pokemons_captured: 2,
							poke_stop_visits: 0,
							pokeballs_throws: 0,
							pokemon_caught_by_type: Int32Array.from([]),
							small_rattata_caught: 0
						}
					}
				}
			];

			for i in [1..151]
				data.inventory_delta.inventory_items.push({
						inventory_item_data: {
							pokemon_data: {
								id: i,
								pokemon_id: i,
								cp: 10,
								stamina: 10,
								stamina_max: 10,
								move_1: "QUICK_ATTACK_FAST",
								move_2: "AIR_CUTTER",
								height_m: 0.29903167486190796,
								weight_kg: 1.878556489944458,
								individual_attack: 1,
								individual_defense: 14,
								individual_stamina: 15,
								cp_multiplier: 0.09399999678134918,
								pokeball: "ITEM_POKE_BALL",
								captured_cell_id: "5082605749092220928",
								creation_time_ms: "1468915294161"
							}
						}
					});
		data

I am not even sure if all of this is required.. But its pretty straightforward.. Replace inventory_items with array containing just PlayerStats, then add 151 Pokemons with ids 1-151 and be fone with it!

The end result is.. well the same as the previous as I was too lazy to run the initial version of replacePokemons to get the real initial result sooo.. Now we’re left with just taking a screenshot of every Pokémon.. Should be easy.. right!

For our good you can scroll the list of Pokémons by swiping so capturing screenshot should be easy as taking screenshot with Pokémon id as name, then swipe predefined amount, sleep 10 seconds to let the model load, and repeat!

A little Windows shell script I made is as follows:

FOR /L %%G IN (1,1,151) DO (
	adb screencap -p /sdcard/%%G.png
	adb pull /sdcard/%%G.png
	adb shell input touchscreen swipe 400 350 200 350
	ping 127.0.0.1 -n 11 > nul
)

pause

Its pretty simple!

  • Repeat from 1 to 151
    • Take a screenshot with naming it with looper number
    • Pull the screenshot from the sdcard to the working directory
    • Evaluate a swipe, from right to left, changing the Pokémon on the screen
    • Sleep for ten seconds, because this is not implemented in a simple sleep fashion, ping 11 times and get approximately a 10 second delay

After running this for.. let me calculate.. approx 30 minutes.. we’re done, and have a folder full of images!

pokimons

As you can see, we’re good, and as far as this glimpse tells me.. we did fine! Now its time to get on with these pictures.. but that’s a topic for another blog.. Thanks for reading!

Automating patching of Pokemon Go – and other applications..

Hello guys!

Last time I posted about patching Pokemon Go without touching the native side, posted it on reddit and gained some good upvotes in /r/pokemongodev, and initial thought in the post was, should this be automated, and given the popularity, I’d say hell yeah!

That’s what I have been up for previous couple of days, writing a fool proof command line based application that lets users patch APK’s with any set of changes, currently it only supports complete file replacements or additions, but that’s easily expandable if required.

The basic task the tool was geared towards is to use APK Tool, as runnable Jar to unpack the APK, then apply some kind of patches to the unpacked folder and repack, sign and align it.

Requirements of running the tool is to have Android SDK, APK Tool and a JDK Installation, which are pretty common, if you’re into this kind of stuff!

The setup is quite simple, when you run the released jar from command line, the program looks for ANDROID_HOME environment variable, JVM parameter and the configuration file, created after the first run to see if its been set, and if it hasn’t it asks user to input it.

ANDROID_HOME here is the root folder of Android SDK, which could automatically be detected to some extent, but as I have other things to do.. for now.. It just looks for the environment variable and command line flag.. You can see where Android SDK is located from Android studio, but at least in Windows environment its usually located in

C:\Users\PasiMatalamaki\AppData\Local\Android\Sdk

Where PasiMatalamaki is replaced with your epic us3rname.

Aftere setting up ANDROID_HOME, we’re not done yet.. The next thing to set is path to APK Tool.jar which can be downloaded from here. Just place it anywhere and copy the path to the jar and paste it in.. after that is the final step, before getting to the real deal! The most important part, which hopefully you’ve already set! Setting JAVA_HOME environment variable or just inputting it..

As written in the first Google result, If you didn’t change the path during installation, it will be something like this: C:\Program Files\Java\jdk1.8.0_65

And that is the final step before getting into the tool, after that you will be asked for build tools version you’d like to use, if you’ve got multiple.

All should work, but I am myself using the newest one, this is other thing, that could be modified to automatically pick the newest one, but man.. the effort!

Choose your buildToolsDir
0: 19.1.0
1: 21.1.2
2: 23.0.1
3: 23.0.3
4: 24.0.0
5: 24.0.1

So I input 5 and I’ll be asked for the apk to be used, or if the folder you’re working in, as in the folder you ran the jar in, contains none, you will be asked for a directory where the apk is, or just a full path to the apk.

Choose APK to be used
0: com.nianticlabs.pokemongo(version: 0.35.0, name: pokemon_original.apk)

For example here’s only one apk, which is the pokemon go, as used in this example, so we select 0.

After that the application will list available patches in the given directory, which by default is the /patches folder, which by default will look like this

Toggle selected patches, or press enter to proceed patching..
0: [ ] frida_gadget
Add Frida Gadget

Adds Frida Gadget to libs and loads it, easy!
1: [ ] remove_pinning
Remove Certificate Pinning

These are the two directories inside the /patches folder, which contains the following structure:

  • patch_name:
    • details.txt
    • replaced files…

as example there two patches in the release. The patching simply replaces the files inside the apk by the files inside the patch, simple and efficient!

You can toggle the patches by their version number, for example, here we want to apply remove pinning patch, we just input 1, and after that we can press enter to proceed with patching.

After that we’ll see a long output of information of whats happening, but after its all done, we’ll see output file

build_output_release.apk

which is result of debuilding, patching, rebuilding, signing and aligning, which is ready to be installed on an Android device! Simply run

adb install build_output_release.apk

And enjoy your Pokemon Go without certificate pinning!

Thanks for reading and for those interested, be sure to see the project in GitHub!

Removing Certificate Pinning from Pokemon Go Without Going Native

What is up?

Today we’re removing certificate pinning from Pokemon Go to man-in-the-middle the connection between my phone and the server, which is now required.

Main difference from similar posts, such as Eaton’s method where he removes it from the native code, is that, we’re not stepping to the dark side, and the main reason for that, for me, its much easier to patch the applications Dalvik code than the native ARM code, and I don’t feel too comfortable poking around in the native code, also I don’t have the tool required, Ida Pro with ARM..

Anyway! Lets get to the real deal, right!

First step, we need to take apart the APK, which is simple! Just use your favourite APK Extractor from Google Play, dump the APK on your machine and use APK Tools to unpack it.

java -jar apktool_2.1.1.jar d pokemon_original.apk -o pokemon

Where apktool_2.1.1.jar is your flavour of APK tool and pokemon_original.apk is the APK dumped from your device.

After that we have a folder containing all the files, good to modify!

Logically we make our way through niantic packages, com/nianticlabs/nia/network sounds good, right?

In this package we find a NianticTrustManager.smali, so its time to fire up your favorite text editor, you can use any, but please don’t use Notepad, its pain in the ass to read..

Given the name too, its clear that this Class implements X509TrustManager, which is as the name says, responsible for the trust management. So its time to lose the trust, right?

Because I’ve poked around with this before, I know how to overcome this slight problem, we’ll have just to return a null from getAcceptedIssuers() and do nothing in checkClientTrusted() and checkServerTrusted()

We take roughly our handle of the latter two methods and remove all the code, leaving behind this kind of structure.

# virtual methods
.method public checkClientTrusted([Ljava/security/cert/X509Certificate;Ljava/lang/String;)V
    .locals 2
    .param p1, "chain"    # [Ljava/security/cert/X509Certificate;
    .param p2, "authType"    # Ljava/lang/String;
    .annotation system Ldalvik/annotation/Throws;
        value = {
            Ljava/security/cert/CertificateException;
        }
    .end annotation

    .prologue
    return-void
.end method

.method public checkServerTrusted([Ljava/security/cert/X509Certificate;Ljava/lang/String;)V
    .locals 2
    .param p1, "chain"    # [Ljava/security/cert/X509Certificate;
    .param p2, "authType"    # Ljava/lang/String;
    .annotation system Ldalvik/annotation/Throws;
        value = {
            Ljava/security/cert/CertificateException;
        }
    .end annotation

    .prologue
    return-void
.end method

And in the getAcceptedIssuers() we have to actually write some code, meh.. Shouldn’t be too hard!

.method public getAcceptedIssuers()[Ljava/security/cert/X509Certificate;
    .locals 2

    .prologue
    .line 44
    const/4 v0, 0x0

    return-object v0
.end method

So we just assign v0 a null(0) and return it with no further ado

Okay, I think we are ready.. Lets cross our fingers and put together this goodness!

First we build the apk from the folder that contains all the files, meaning the pokemon folder!

java -jar apktool_2.1.1.jar b pokemon -o pokemon.apk

Then we need to get a keystore from somewhere, which will be used for signing, creating that is quite simple, just run the keytool!

keytool -genkey -v -keystore key -alias pokego -keyalg RSA -keysize 2048 -validity 10000

After that we’re left with a key that is used to sign the actual apk using jarsigner!

jarsigner -verbose -keystore key -signedjar "pokemon_ua.apk" "pokemon.apk" pokego -storepass SecretPokeGoPassXD

This creates a pokemon_ua.apk file. ua stands for Un-Aligned, if you’re into Android Dev, you might now that the apks needs to be aligned before shipping! Which is a matter of running a zipalign on them, which is also described in the manual signing process article.

zipalign -f 4 "pokemon_ua.apk" "pokemon_release.apk"

And finally we got the final ‘release’ build! Lets shove that into our device! Make sure you uninstall previous installations of the app!

adb install pokemon_release.apk

Run the app on the device, throw in your age, sign in aaand… In like a flynn!

Open up your favorite MITM proxy, such as Charles, which is what I am going to use, and configure your device to connect to the proxy instead, continue using the app and you will see what will make angels hum.

result

 

 

 

There we go, unencrypted version of the packets! However this is in a undecoded protocol buffer, which descriptor you can probably find online.. In Github or somewhere..!