X-Git-Url: https://git.based.quest/?a=blobdiff_plain;f=content%2Fposts%2Freverse-engineering-a-mobile-app-protobuf-api.md;h=2f412e48f0fd18529711ba77e62b79f789ec0976;hb=72731d2bb816f4800a91fb3adcb1ea99af5f401e;hp=c28e6e4e7de1aad1e3ac193feaaa42cb669a4a56;hpb=2a5bb5a5452d7eaf1570b96b68d8d5a3e504c5bc;p=web-hugo.git diff --git a/content/posts/reverse-engineering-a-mobile-app-protobuf-api.md b/content/posts/reverse-engineering-a-mobile-app-protobuf-api.md index c28e6e4..2f412e4 100644 --- a/content/posts/reverse-engineering-a-mobile-app-protobuf-api.md +++ b/content/posts/reverse-engineering-a-mobile-app-protobuf-api.md @@ -16,12 +16,17 @@ is often left behind in source format within client applications. In this series of blogposts, I will be using a mobile game "Egg, Inc." as the target for demonstration. It's a simple time killer app that got me through boring long waits when I was still at school. -I won't bore you with details, but in essence it's an incremental game with partial online features -such as cloud save, co-op contracts and server scheduled boosts. +Egg, Inc. is a basic incremental "idler" game where your goal is to take over the world food supply with ever-increasing supply of eggs, +if you have ever played Cookie Clicker, you know the premise of something like that. You have to unlock denser and denser eggs - the game +is also designed around the fact that you can do certain online-tied activites such as Contracts to unlock more Soul Eggs (prestige boost) and +"Eggs of Prophecy" which increase potency of your Soul Eggs. + +It's rather simple game with a very minimal API, making it perfect for learning. You may not like the game, but that's beside the point. +The simplicity of our target matters here. ## The existing works -In some cases, you will find previous works on the target you pick, in my case, some clever people have created -[scripts to extract .proto file out of app.](https://github.com/DavidArthurCole/EggIncProtoExtractor) +In some cases, you will find previous works on the target you pick. In my case, some clever people have created +[scripts to extract .proto file out of the app.](https://github.com/DavidArthurCole/EggIncProtoExtractor) I advise you to check it out if you wish to get a better understanding of how you would go about retrieving the API spec .proto file for your target. @@ -30,6 +35,13 @@ Further there are a few dedicated individuals in the game's community who have c For this blog purposes, we will assume the game server is shut down (as in we cannot query from the live API) and our goal is to make a semi-functional selfhosted gameserver for our own needs, assuming we are the only one on said server. +## How to source builds of a game +There are two methods of sourcing the apk file here - one method is if you already have the app installed, install something like ZArchiver +and extract it from /data/app/ - identifying the app by its icon. From there you will find `base.apk` which is enough for most apps. + +Alternatively, if the app is still available on Google Play, you can use an app like Aurora Store to go to the store detail page, select +"Manual Download" and enter a known Build ID. + ## Getting Started Thanks to the previously mentioned script, it's easy to get started - find the APK, extract protobuf spec file, convert it with protoc and we're done there. One small problem - due to cheaters, latest version of the game includes "AuthenticatedMessage" structure, @@ -39,7 +51,7 @@ At this point, after a bit of internal dilemma, I decided to not further the pro more morally sound decision of picking a version prior to these integrity checks. We can crack that another day as all the needed information is retained in the app itself. -Going forward with this, we are targetting game version 1.12.13 (Build ID 111121). +Going forward with this, we are targetting game version 1.12.13 (Build ID 111121 - use that in Aurora Store). With all that out of the way, lets get into actual commands used here: ``` @@ -83,17 +95,16 @@ When I jumped to it, I saw an exactly adjacent string to it which could give mor Interesting, `www.auxbrain.com`. If we jump to its XREF, we get a garbled function, but what it seems to be doing is setting up certain global values. -So we have a potential API endpoint, let's put it to the test. We're not going to recompile anything yet or do any byte-patching, -let's try a quick smoke-test. Ensure your phone is rooted and you have a variant of Xposed Framework installed (I used LSPosed). -We will need to untrip the SSL pinning present in most apps, including this one, I used [io.github.tehcneko.sslunpinning](https://github.com/Xposed-Modules-Repo/io.github.tehcneko.sslunpinning) module. -(Note: I know it is possible to repackage the app to do SSL unpinning in most cases, but in many cases, you won't know if it's worth the effort yet) +## The smoke-test -Next, install [AdAway app from F-Droid](https://f-droid.org/packages/org.adaway/) so we can setup a redirection on any network we are on. +So we have a potential API endpoint, let's put it to the test. We can do a quick smoke test by setting up a webserver. + +Install [AdAway app from F-Droid](https://f-droid.org/packages/org.adaway/) so we can setup a redirection on any network we are on. Inside AdAway, add a redirection rule for the address we just found and point it to an IP address in your LAN that will run the API server. -Generate a self-signed certificate authority and a certificate signed by it and run a webserver with both HTTP and HTTPS on the API server machine. -Import the self-signed CA to your phone's truststore. Once all of that is done, run the app for first time. +(NOTE: AdAway doesn't detect any subdomains nor can it do wildcard, you will need to include the FQDN of the API endpoint `www.auxbrain.com`) +Once you're done setting up the redirection, run any webserver such as nginx for a quick and dirty test. ``` 192.168.1.212 - - [...] "POST /ei/first_contact HTTP/1.1" 404 0 "-" ``` @@ -380,7 +391,7 @@ GENEROUS DRONES ``` This looks promising, right off the bat, first strings I'd check here are `r_icon_drone_rewards`, `b_icon_drone_boost`, `drone-boost` and `GENEROUS DRONES`. -I inspected all 4 of them, and when I got to final 2, I found the enum string translations used for event IDs - here they are extracted for game version 1.12.13 +I inspected all 4 of them, and when I got to the final 2, I found the enum string translations used for event IDs - here they are extracted for game version 1.12.13 ``` piggy-boost (Rate piggy fills is increased.) piggy-cap-boost (UNLIMITED PIGGY;Gains are retained when event ends.) @@ -428,7 +439,7 @@ At this point, we can start dog-fooding the project. Lets start with whatever ba ### Contracts As we progress the game and start performing prestiges, we unlock a feature called "Contracts" - but disaster strikes as we don't have any contracts we could -accept. So far we stil see our good friends `/ei/get_periodicals` and `/ei/save_backup` hammering the server at regular intervals. +accept. So far we still see our good friends `/ei/get_periodicals` and `/ei/save_backup` hammering the server at regular intervals. When we created the periodicals response payload, you might have noticed in the protobuf message an optional field called `ContractsResponse contracts`. Lets see what this ContractsResponse message contains. @@ -487,7 +498,7 @@ beginners will get easier contract goals while more advanced players get harder We can put two-and-two together here and infer that `repeated Goal goals` is the legacy contract system - where everyone was on equal footing and `repeated GoalSet goal_sets` is the *new* goal system that is split into Standard and Elite. -We also learn that in future game version, they completely reworked how contracts work *yet* again into a grading "bracket" system. Fortunately, +We also learn that in future game versions, they completely reworked how contracts work *yet* again into a grading "bracket" system. Fortunately, we do not have to worry about that in our current target revision. Now to get the ball rolling, there is conveniently a starting point set ahead for us already. The developer of game intended to ease new players into @@ -541,7 +552,7 @@ This page tells us all the information we need to compose our first Contract, so return base64.b64encode(CurrentPeriodicals.SerializeToString()) ``` -Lets try that out in-game now - after waiting for a minute, we see our contract prop up, but I immediately noticed one thing amiss. +Lets try that out in-game now - after waiting for a minute, we see our contract pop up, but I immediately noticed one thing amiss. The contract goals are swapped! I am getting Elite contract rewards for a Standard contract. This piece of information now tells us that the first entry in GoalSets refers to Elite rewards and the second entry in GoalSets to Standard rewards. @@ -568,5 +579,7 @@ about the project you are currently doing - refactoring becomes an essential par I won't give any promises for a part 2 any time soon, but I will be trying to make this feature complete, so without further ado, here are the git repository links: [github.com](https://github.com/cernodile/reEgg), [git.based.quest](https://git.based.quest/?p=reEgg.git;a=tree;h=refs/heads/master;hb=refs/heads/master). +Next time we will dive into apps that use SSL/TLS and making onboarding for your friends easier. + Thank you for reading and making it all the way to the end, - Cernodile