From: Joann Mõndresku Date: Sun, 12 May 2024 09:06:36 +0000 (+0300) Subject: Fixes and extra info + rootless SSL unpinning X-Git-Url: https://git.based.quest/%7B%7B.Site.BaseURL%7D%7D?a=commitdiff_plain;h=1e75f3554b97596b82ba933e1296d133b3f9d9f0;p=web-hugo.git Fixes and extra info + rootless SSL unpinning --- diff --git a/content/posts/reverse-engineering-a-mobile-app-protobuf-api.md b/content/posts/reverse-engineering-a-mobile-app-protobuf-api.md index c28e6e4..5c94bd1 100644 --- a/content/posts/reverse-engineering-a-mobile-app-protobuf-api.md +++ b/content/posts/reverse-engineering-a-mobile-app-protobuf-api.md @@ -16,12 +16,17 @@ is often left behind in source format within client applications. In this series of blogposts, I will be using a mobile game "Egg, Inc." as the target for demonstration. It's a simple time killer app that got me through boring long waits when I was still at school. -I won't bore you with details, but in essence it's an incremental game with partial online features -such as cloud save, co-op contracts and server scheduled boosts. +Egg, Inc. is a basic incremental "idler" game where your goal is to take over the world food supply with ever-increasing supply of eggs, +if you have ever played Cookie Clicker, you know the premise of something like that. You have to unlock denser and denser eggs - the game +is also designed around the fact that you can do certain online-tied activites such as Contracts to unlock more Soul Eggs (prestige boost) and +"Eggs of Prophecy" which increase potency of your Soul Eggs. + +It's rather simple game with a very minimal API, making it perfect for learning. You may not like the game, but that's beside the point. +The simplicity of our target matters here. ## The existing works -In some cases, you will find previous works on the target you pick, in my case, some clever people have created -[scripts to extract .proto file out of app.](https://github.com/DavidArthurCole/EggIncProtoExtractor) +In some cases, you will find previous works on the target you pick. In my case, some clever people have created +[scripts to extract .proto file out of the app.](https://github.com/DavidArthurCole/EggIncProtoExtractor) I advise you to check it out if you wish to get a better understanding of how you would go about retrieving the API spec .proto file for your target. @@ -30,6 +35,13 @@ Further there are a few dedicated individuals in the game's community who have c For this blog purposes, we will assume the game server is shut down (as in we cannot query from the live API) and our goal is to make a semi-functional selfhosted gameserver for our own needs, assuming we are the only one on said server. +## How to source builds of a game +There are two methods of sourcing the apk file here - one method is if you already have the app installed, install something like ZArchiver +and extract it from /data/app/ - identifying the app by its icon. From there you will find `base.apk` which is enough for most apps. + +Alternatively, if the app is still available on Google Play, you can use an app like Aurora Store to go to the store detail page, select +"Manual Download" and enter a known Build ID. + ## Getting Started Thanks to the previously mentioned script, it's easy to get started - find the APK, extract protobuf spec file, convert it with protoc and we're done there. One small problem - due to cheaters, latest version of the game includes "AuthenticatedMessage" structure, @@ -39,7 +51,7 @@ At this point, after a bit of internal dilemma, I decided to not further the pro more morally sound decision of picking a version prior to these integrity checks. We can crack that another day as all the needed information is retained in the app itself. -Going forward with this, we are targetting game version 1.12.13 (Build ID 111121). +Going forward with this, we are targetting game version 1.12.13 (Build ID 111121 - use that in Aurora Store). With all that out of the way, lets get into actual commands used here: ``` @@ -85,7 +97,7 @@ certain global values. So we have a potential API endpoint, let's put it to the test. We're not going to recompile anything yet or do any byte-patching, let's try a quick smoke-test. Ensure your phone is rooted and you have a variant of Xposed Framework installed (I used LSPosed). -We will need to untrip the SSL pinning present in most apps, including this one, I used [io.github.tehcneko.sslunpinning](https://github.com/Xposed-Modules-Repo/io.github.tehcneko.sslunpinning) module. +We will need to unarm the SSL pinning present in most apps, including this one, I used [io.github.tehcneko.sslunpinning](https://github.com/Xposed-Modules-Repo/io.github.tehcneko.sslunpinning) module. (Note: I know it is possible to repackage the app to do SSL unpinning in most cases, but in many cases, you won't know if it's worth the effort yet) Next, install [AdAway app from F-Droid](https://f-droid.org/packages/org.adaway/) so we can setup a redirection on any network we are on. @@ -380,7 +392,7 @@ GENEROUS DRONES ``` This looks promising, right off the bat, first strings I'd check here are `r_icon_drone_rewards`, `b_icon_drone_boost`, `drone-boost` and `GENEROUS DRONES`. -I inspected all 4 of them, and when I got to final 2, I found the enum string translations used for event IDs - here they are extracted for game version 1.12.13 +I inspected all 4 of them, and when I got to the final 2, I found the enum string translations used for event IDs - here they are extracted for game version 1.12.13 ``` piggy-boost (Rate piggy fills is increased.) piggy-cap-boost (UNLIMITED PIGGY;Gains are retained when event ends.) @@ -428,7 +440,7 @@ At this point, we can start dog-fooding the project. Lets start with whatever ba ### Contracts As we progress the game and start performing prestiges, we unlock a feature called "Contracts" - but disaster strikes as we don't have any contracts we could -accept. So far we stil see our good friends `/ei/get_periodicals` and `/ei/save_backup` hammering the server at regular intervals. +accept. So far we still see our good friends `/ei/get_periodicals` and `/ei/save_backup` hammering the server at regular intervals. When we created the periodicals response payload, you might have noticed in the protobuf message an optional field called `ContractsResponse contracts`. Lets see what this ContractsResponse message contains. @@ -487,7 +499,7 @@ beginners will get easier contract goals while more advanced players get harder We can put two-and-two together here and infer that `repeated Goal goals` is the legacy contract system - where everyone was on equal footing and `repeated GoalSet goal_sets` is the *new* goal system that is split into Standard and Elite. -We also learn that in future game version, they completely reworked how contracts work *yet* again into a grading "bracket" system. Fortunately, +We also learn that in future game versions, they completely reworked how contracts work *yet* again into a grading "bracket" system. Fortunately, we do not have to worry about that in our current target revision. Now to get the ball rolling, there is conveniently a starting point set ahead for us already. The developer of game intended to ease new players into @@ -541,7 +553,7 @@ This page tells us all the information we need to compose our first Contract, so return base64.b64encode(CurrentPeriodicals.SerializeToString()) ``` -Lets try that out in-game now - after waiting for a minute, we see our contract prop up, but I immediately noticed one thing amiss. +Lets try that out in-game now - after waiting for a minute, we see our contract pop up, but I immediately noticed one thing amiss. The contract goals are swapped! I am getting Elite contract rewards for a Standard contract. This piece of information now tells us that the first entry in GoalSets refers to Elite rewards and the second entry in GoalSets to Standard rewards. @@ -553,6 +565,68 @@ Now, the above code could be a lot neater. For your homework, if you're not skip a contract database and try scheduling them like the game originally did - a "Leggacy" contract every Friday and regular contracts showing up every 1-2 weeks for roughly 2 weeks. +## Rootless SSL Unpinning + Endpoint URL patching +Let's make the app not require a VPN or root privileges - let's make user CAs work and the endpoint URL something we control on the public net. +Start off by pulling the following repository +``` +git clone https://github.com/ilya-kozyr/android-ssl-pinning-bypass.git +python3 -m venv .venv +source .venv/bin/activate +pip install -r requirements.txt +cp /path/to/your/apk . +python3 apk-rebuild.py egginc.apk --pause +``` + +Open a new terminal window, the script will wait for us to perform modifications, enter the created folder `egginc.apk-decompiled` and `lib`. + +We have two folders here now, `arm64-v8a` and `armeabi-v7a`, just as we saw when we pulled the .so file out of the apk earlier. Let's tackle +the 64-bit build first. + +For arm64 build it was really simple to perform bytepatch on the said endpoint. We already know it's supposed to look as `G?www.auxbrain.com` - let's probe the .so library a bit. +``` +$> hexdump -C libegginc.so | grep "ww.auxbrain.co" -A2 -B2 +00b02b40 cd cc 4c 3f 00 00 00 00 00 00 00 00 00 00 80 3f |..L?...........?| +00b02b50 00 00 00 00 00 00 00 00 00 00 00 00 14 ae 47 3f |..............G?| +00b02b60 77 77 77 2e 61 75 78 62 72 61 69 6e 2e 63 6f 6d |www.auxbrain.com| +00b02b70 00 48 54 54 50 20 52 45 51 3a 20 25 64 00 64 61 |.HTTP REQ: %d.da| +00b02b80 74 61 3d 00 65 69 2f 66 69 72 73 74 5f 63 6f 6e |ta=.ei/first_con +``` + +We seem to have nothing blocking our way, let's create hex representations of `G?www.auxbrain.com` and a target domain of equal length, for example `G?eggs.based.quest`. + +(Note: You can choose a shorter name as well, if you null-terminate the extra bytes as padding) +``` +$> echo "G?www.auxbrain.com" | hexdump -ve '1/1 "%.2X"' +473F7777772E617578627261696E2E636F6D0A +$> echo "G?eggs.based.quest" | hexdump -ve '1/1 "%.2X"' +473F656767732E62617365642E71756573740A +``` + +Remove the trailing `0A` from end of both hex strings and now proceed as follows: +``` +# Place the source in first bracket of sed and the new URL at second bracket. +hexdump -ve '1/1 "%.2X"' libegginc.so | sed "s/473F7777772E617578627261696E2E636F6D/473F656767732E62617365642E7175657374/g" | xxd -r -p > patched.so +``` + +Huzzah! We now have a patched linked-library for the arm64 build. Let's also patch the 32-bit version. +``` +$> hexdump -C libegginc.so | grep "ww.auxbrain.co" -A2 -B2 +0087b770 69 67 68 5f 74 6f 6f 5f 6d 61 6e 79 5f 70 78 00 |igh_too_many_px.| +0087b780 74 61 62 6c 65 74 5f 68 64 70 69 00 00 00 00 00 |tablet_hdpi.....| +0087b790 77 77 77 2e 61 75 78 62 72 61 69 6e 2e 63 6f 6d |www.auxbrain.com| +0087b7a0 00 00 00 00 00 00 00 00 65 69 2f 66 69 72 73 74 |........ei/first| +0087b7b0 5f 63 6f 6e 74 61 63 74 00 00 00 00 00 00 00 00 |_contact........| +``` +This one lacks the `G?` prefix on API endpoint, but we still have null terminators we can rely on. Let's replace the `473F` from our previous strings with `0000`. +``` +# Place the source in first bracket of sed and the new URL at second bracket. +hexdump -ve '1/1 "%.2X"' libegginc.so | sed "s/00007777772E617578627261696E2E636F6D/0000656767732E62617365642E7175657374/g" | xxd -r -p > patched.so +``` + +Replace both of the libegginc.so files with the patched.so files. Move back to main terminal window and press ENTER. + +We now have a patched and debug signed apk for the game that isn't SSL pinned and contains a custom API endpoint we control without a VPN. + ## Conclusion so far We have created a (rather ugly looking) server emulator for the game. It functions, but it needs a lot of work still before we can call it ready. If you have followed this far, give yourself pat on the back - if you actually tried to run this code, give yourself an extra pat on the back.