| 1 | --- |
| 2 | title: "Reverse Engineering a Mobile App Protobuf API" |
| 3 | date: 2024-05-11T12:00:00+03:00 |
| 4 | description: "In recent times more than ever, live service and overuse of APIs soon-to-be-stale is only increasing. This causes a lot of once-written software to become either unusable or handicapped in many regards. One way to fight this is to learn reverse engineering for sake of digital preservation. In this blog, I take you through a journey of reverse engineering a simple mobile game's protobuf API." |
| 5 | tags: ['tutorial', 'reverse-engineering', 'opensource'] |
| 6 | type: blog |
| 7 | draft: false |
| 8 | --- |
| 9 | |
| 10 | # Why |
| 11 | Why not? Digital preservation is important, even if you don't care for a specific program. |
| 12 | This is also a good way to get started with protocol reverse engineering due to way protobuf |
| 13 | is often left behind in source format within client applications. |
| 14 | |
| 15 | ## The target |
| 16 | In this series of blogposts, I will be using a mobile game "Egg, Inc." as the target for |
| 17 | demonstration. It's a simple time killer app that got me through boring long waits when I was still at school. |
| 18 | |
| 19 | Egg, Inc. is a basic incremental "idler" game where your goal is to take over the world food supply with ever-increasing supply of eggs, |
| 20 | if you have ever played Cookie Clicker, you know the premise of something like that. You have to unlock denser and denser eggs - the game |
| 21 | is also designed around the fact that you can do certain online-tied activites such as Contracts to unlock more Soul Eggs (prestige boost) and |
| 22 | "Eggs of Prophecy" which increase potency of your Soul Eggs. |
| 23 | |
| 24 | It's rather simple game with a very minimal API, making it perfect for learning. You may not like the game, but that's beside the point. |
| 25 | The simplicity of our target matters here. |
| 26 | |
| 27 | ## The existing works |
| 28 | In some cases, you will find previous works on the target you pick. In my case, some clever people have created |
| 29 | [scripts to extract .proto file out of the app.](https://github.com/DavidArthurCole/EggIncProtoExtractor) |
| 30 | I advise you to check it out if you wish to get a better understanding of how you would go about retrieving the |
| 31 | API spec .proto file for your target. |
| 32 | |
| 33 | Further there are a few dedicated individuals in the game's community who have created numerous tools and historical databases. |
| 34 | |
| 35 | For this blog purposes, we will assume the game server is shut down (as in we cannot query from the live API) and our goal is |
| 36 | to make a semi-functional selfhosted gameserver for our own needs, assuming we are the only one on said server. |
| 37 | |
| 38 | ## How to source builds of a game |
| 39 | There are two methods of sourcing the apk file here - one method is if you already have the app installed, install something like ZArchiver |
| 40 | and extract it from /data/app/ - identifying the app by its icon. From there you will find `base.apk` which is enough for most apps. |
| 41 | |
| 42 | Alternatively, if the app is still available on Google Play, you can use an app like Aurora Store to go to the store detail page, select |
| 43 | "Manual Download" and enter a known Build ID. |
| 44 | |
| 45 | ## Getting Started |
| 46 | Thanks to the previously mentioned script, it's easy to get started - find the APK, extract protobuf spec file, convert it with |
| 47 | protoc and we're done there. One small problem - due to cheaters, latest version of the game includes "AuthenticatedMessage" structure, |
| 48 | which contains a salted sha256sum of the payload message. |
| 49 | |
| 50 | At this point, after a bit of internal dilemma, I decided to not further the problem while service is still live for people playing and did the |
| 51 | more morally sound decision of picking a version prior to these integrity checks. We can crack that another day as all the needed information |
| 52 | is retained in the app itself. |
| 53 | |
| 54 | Going forward with this, we are targetting game version 1.12.13 (Build ID 111121 - use that in Aurora Store). |
| 55 | |
| 56 | With all that out of the way, lets get into actual commands used here: |
| 57 | ``` |
| 58 | git clone https://github.com/DavidArthurCole/EggIncProtoExtractor.git |
| 59 | cd EggIncProtoExtractor |
| 60 | ./apkextract.sh com.auxbrain.egginc_1.12.13.apk |
| 61 | # We should have a new folder "protos" now with resulting files |
| 62 | cd protos |
| 63 | # There should be a file called ei.proto - that's our protobuf spec file |
| 64 | # At this point, we can use the protoc utility which can convert the specfile |
| 65 | # to interfaces in C++, C#, Java, Kotlin, Objective-C, PHP, Python and Ruby with |
| 66 | # additional plugin support for Dart and Go. |
| 67 | # To make this easier to understand, we will use Python in this demonstration |
| 68 | protoc -I=. --python_out=. ./ei.proto |
| 69 | # Success! We now have a "ei_pb2.py" file which can be directly imported to Python programs |
| 70 | ``` |
| 71 | |
| 72 | With the protobuf interface in Python created, we can now proceed with creating the API emulator - but there's a slight problem. |
| 73 | What URL? What endpoints? How do we find this out? Simple answer, disassembling the game. Get your RE tool of choice, I will be |
| 74 | using [Ghidra](https://ghidra-sre.org/) myself. |
| 75 | |
| 76 | (Note: You can also just try to find this out using tools such as WireShark) |
| 77 | |
| 78 | The game contains a linked-library written in C++, which you can find inside the .apk `lib` folder, named as `libegginc.so`. |
| 79 | This is perfect for our use-case, Ghidra is going to slice through this like butter. Import the file to your RE tool of choice |
| 80 | and let it perform some analysis on it, have a cup of tea or coffee as this is going to take a hot minute. |
| 81 | |
| 82 | Once that's done, we are going to start by looking at the defined strings - try our luck there. Search for any debug prints left |
| 83 | behind or maybe some clues. I started by searching for `http`, which lead me to following string `"HTTP REQ: %d"`, seems promising. |
| 84 | When I jumped to it, I saw an exactly adjacent string to it which could give more clues: |
| 85 | ``` |
| 86 | s_www.auxbrain.com_00c02b60 XREF[0,1]: FUN_00518ab8:00518b38(R) |
| 87 | 00c02b5e 47 3f 77 ds "G?www.auxbrain.com" |
| 88 | 77 77 2e |
| 89 | 61 75 78 |
| 90 | s_HTTP_REQ:_%d_00c02b71 XREF[1]: makeRequestInternal:0067bbd4(*) |
| 91 | 00c02b71 48 54 54 ds "HTTP REQ: %d" |
| 92 | 50 20 52 |
| 93 | 45 51 3a |
| 94 | ``` |
| 95 | Interesting, `www.auxbrain.com`. If we jump to its XREF, we get a garbled function, but what it seems to be doing is setting up |
| 96 | certain global values. |
| 97 | |
| 98 | ## The smoke-test |
| 99 | |
| 100 | So we have a potential API endpoint, let's put it to the test. We're not going to recompile anything yet or do any byte-patching, |
| 101 | let's try a quick smoke-test. Ensure your phone is rooted and you have a variant of Xposed Framework installed (I used LSPosed). |
| 102 | We will need to unarm the SSL pinning present in most apps, including this one, I used [io.github.tehcneko.sslunpinning](https://github.com/Xposed-Modules-Repo/io.github.tehcneko.sslunpinning) module. |
| 103 | (NOTE: Users without root might want to skip to end of article where I showcase unpinning the app manually) |
| 104 | |
| 105 | Next, install [AdAway app from F-Droid](https://f-droid.org/packages/org.adaway/) so we can setup a redirection on any network we are on. |
| 106 | Inside AdAway, add a redirection rule for the address we just found and point it to an IP address in your LAN that will run the API server. |
| 107 | |
| 108 | Generate a self-signed certificate authority and a certificate signed by it and run a webserver with both HTTP and HTTPS on the API server machine. |
| 109 | ``` |
| 110 | # Create an ext file containing the Subject Alternative Name (SAN) |
| 111 | # DNS.1 should correspond to the API endpoint of the app. |
| 112 | # NOTE! If you are changing the API endpoint to a public domain, you can just use a public cert, no need for any of this. |
| 113 | cat > auxbrain.ext << EOF |
| 114 | authorityKeyIdentifier=keyid,issuer |
| 115 | basicConstraints=CA:FALSE |
| 116 | keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment |
| 117 | subjectAltName = @alt_names |
| 118 | |
| 119 | [alt_names] |
| 120 | DNS.1 = www.auxbrain.com |
| 121 | EOF |
| 122 | |
| 123 | # Create your own Certificate Authority |
| 124 | openssl genrsa -des3 -out myCA.key 2048 |
| 125 | openssl req -x509 -new -nodes -key myCA.key -sha256 -days 1825 -out myCA.pem |
| 126 | # Create a CSR and lets have the new CA sign it |
| 127 | openssl genrsa -out auxbrain.key 2048 |
| 128 | openssl req -new -key auxbrain.key -out auxbrain.csr -nodes |
| 129 | openssl x509 -req -in auxbrain.csr -CA myCA.pem -CAkey myCA.key -CAcreateserial -out auxbrain.crt -days 825 -sha256 -extfile auxbrain.ext |
| 130 | cat auxbrain.crt myCA.pem > auxbrain.pem |
| 131 | # You now have: |
| 132 | # myCA.pem - the public certificate of your root CA |
| 133 | # auxbrain.key - the private key for your webserver |
| 134 | # auxbrain.pem - the public cert for your webserver. |
| 135 | ``` |
| 136 | |
| 137 | Use the generated `auxbrain.pem` and `auxbrain.key` files for your webserver SSL/TLS configuration. For nginx, append following values to your server directive: |
| 138 | ``` |
| 139 | listen 443 ssl; |
| 140 | ssl_certificate /path/to/auxbrain.pem; |
| 141 | ssl_certificate_key /path/to/auxbrain.key; |
| 142 | ssl_session_cache shared:SSL:1m; |
| 143 | ssl_session_timeout 5m; |
| 144 | ssl_ciphers HIGH:!aNULL:!MD5; |
| 145 | ssl_prefer_server_ciphers on; |
| 146 | ``` |
| 147 | |
| 148 | Import the self-signed CA (myCA.pem) to your phone's truststore (Check under your phone's Security/Encryption settings). Once all of that is done, run the app for first time. |
| 149 | |
| 150 | ``` |
| 151 | 192.168.1.212 - - [...] "POST /ei/first_contact HTTP/1.1" 404 0 "-" |
| 152 | ``` |
| 153 | |
| 154 | Bingo. We have contact and we have an API endpoint. Searching for "ei/" in the strings reveals a extensive list of API endpoints, we now have something |
| 155 | to go off from. We have everything we need to start creating the server. |
| 156 | |
| 157 | ## Implementing the Server - Getting first contact |
| 158 | Next up, we create a new project - as we generated the protobuf definitions for Python, we will proceed accordingly. |
| 159 | If you are following along, get respective packages for your operating system to create python venvs. |
| 160 | As the protobufs are being sent over HTTP, we will be serving our application over flask which is being reverse proxied by nginx. |
| 161 | |
| 162 | ``` |
| 163 | # Lets stage the environment |
| 164 | mkdir apiserver |
| 165 | cd apiserver |
| 166 | python -m venv .venv |
| 167 | source .venv/bin/activate |
| 168 | touch app.py |
| 169 | cp ~/EggIncProtoExtractor/protos/ei.proto . |
| 170 | |
| 171 | # Get some dependeices |
| 172 | pip install protobuf |
| 173 | pip install flask |
| 174 | ``` |
| 175 | |
| 176 | We now have the project set up for reading protobuf definitions and a framework to listen for HTTP and routes sent to it. |
| 177 | Let's create an actual listener application, open app.py with your favourite IDE or text editor. |
| 178 | |
| 179 | ``` |
| 180 | import ei_pb2 as EIProto |
| 181 | from flask import Flask |
| 182 | from flask import request |
| 183 | |
| 184 | @app.route("/ei/<path:subpath>", methods=["POST"]) |
| 185 | def ei_routes(subpath): |
| 186 | print("HTTP POST /ei/" + subpath) |
| 187 | print(request.headers) |
| 188 | return "" |
| 189 | ``` |
| 190 | |
| 191 | This should get the ball rolling, we will see whatever call comes in and we can see what the payload of each request contains. |
| 192 | At this point you should setup the reverse proxy, override your nginx / directive with: |
| 193 | ``` |
| 194 | location / { |
| 195 | proxy_pass http://127.0.0.1:5000; |
| 196 | } |
| 197 | ``` |
| 198 | |
| 199 | Reload your nginx and start the flask application you just created with `flask run`. |
| 200 | |
| 201 | Run the app again and have it phone home and see what it contains. |
| 202 | ``` |
| 203 | HTTP POST /ei/first_contact |
| 204 | Host: 127.0.0.1:5000 |
| 205 | Connection: close |
| 206 | Content-Length: 37 |
| 207 | Content-Type: application/x-www-form-urlencoded |
| 208 | User-Agent: Dalvik/2.1.0 (Linux; U; Android 13; M2012K11AG Build/TQ3A.230901.001) |
| 209 | Accept-Encoding: gzip |
| 210 | ``` |
| 211 | |
| 212 | We can see there's a form payload attached to this request, let's modify our app route a bit: |
| 213 | ``` |
| 214 | @app.route("/ei/<path:subpath>", methods=["POST"]) |
| 215 | def ei_routes(subpath): |
| 216 | print("HTTP POST /ei/" + subpath) |
| 217 | print(request.form) |
| 218 | return "" |
| 219 | ``` |
| 220 | |
| 221 | Now if we run the modified flask application again, we see following output on the first_contact endpoint. |
| 222 | ``` |
| 223 | HTTP POST /ei/first_contact |
| 224 | ImmutableMultiDict([('data', 'ChAzNTVlNDZlOTA4OWQxZTRjEAAYAg==')]) |
| 225 | ``` |
| 226 | |
| 227 | We have a base64-encoded protobuf binary data - which isn't terribly useful for reading plain-text, since protobuf *is* a binary |
| 228 | format, so we will need to figure out what protobuf message this payload belongs to. |
| 229 | |
| 230 | Remember that ei.proto file alongside the ei_pb2.py we got earlier? Lets go back there and inspect it a bit. |
| 231 | We know we just contacted something called "first_contact", maybe there is something in that file that could help us? |
| 232 | |
| 233 | ``` |
| 234 | message EggIncFirstContactRequest { |
| 235 | optional string user_id = 1; |
| 236 | optional uint32 client_version = 2; |
| 237 | optional Platform platform = 3; |
| 238 | } |
| 239 | |
| 240 | message EggIncFirstContactResponse { |
| 241 | optional Backup backup = 1; |
| 242 | } |
| 243 | ``` |
| 244 | |
| 245 | Seems like the application is using message names in almost similar fashion to API endpoint names themselves. This will prove |
| 246 | to be useful knowledge. We now know what the payload should be, lets put this to the test. |
| 247 | |
| 248 | Edit your app routine again |
| 249 | ``` |
| 250 | # add "import base64" to top of the file |
| 251 | @app.route("/ei/<path:subpath>", methods=["POST"]) |
| 252 | def ei_routes(subpath): |
| 253 | print("HTTP POST /ei/" + subpath) |
| 254 | if subpath == "first_contact": |
| 255 | # Create the protobuf object so we can load data from the b64 payload |
| 256 | FirstContact = EIProto.EggIncFirstContactRequest() |
| 257 | FirstContact.ParseFromString(base64.b64decode(form["data"])) |
| 258 | print(FirstContact) |
| 259 | else: |
| 260 | print(request.form) |
| 261 | return "" |
| 262 | ``` |
| 263 | |
| 264 | We should now be able to see deserialized output when we run the flask application and the mobile app, let's try it out: |
| 265 | |
| 266 | ``` |
| 267 | HTTP POST /ei/first_contact |
| 268 | user_id: "355e46e9089d1e4c" |
| 269 | client_version: 0 |
| 270 | platform: DROID |
| 271 | ``` |
| 272 | |
| 273 | Nice! We now know how to identify which protobuf object corresponds to which API endpoint. We can now make an educated guess |
| 274 | on what would come next. |
| 275 | |
| 276 | Seeing how we got `EggIncFirstContactRequest` and saw an adjacent `EggIncFirstContactResponse` message in the proto file, we |
| 277 | can safely assume that this is what the game is expecting from us in return. |
| 278 | |
| 279 | Lets modify the server a bit to account for that. |
| 280 | |
| 281 | ``` |
| 282 | @app.route("/ei/<path:subpath>", methods=["POST"]) |
| 283 | def ei_routes(subpath): |
| 284 | print("HTTP POST /ei/" + subpath) |
| 285 | if subpath == "first_contact": |
| 286 | # Create the protobuf object so we can load data from the b64 payload |
| 287 | FirstContact = EIProto.EggIncFirstContactRequest() |
| 288 | FirstContact.ParseFromString(base64.b64decode(form["data"])) |
| 289 | print("We got a first contact hello from user " + FirstContact.user_id) |
| 290 | # Lets respond with a FirstContactResponse |
| 291 | FirstContactResp = EIProto.EggIncFirstContactResponse() |
| 292 | # This takes only one optional argument - a Backup object - but we have no account |
| 293 | # system yet, so we will opt out of sending that for now. |
| 294 | # --- |
| 295 | # We send the payload back as a base64 string - the same way we retrieved it. |
| 296 | return base64.b64encode(FirstContactResp.SerializeToString()) |
| 297 | else: |
| 298 | print(request.form) |
| 299 | return "" |
| 300 | ``` |
| 301 | |
| 302 | Now when we run the app again, we notice that we no longer get spammed this endpoint, but instead in its place we see a few new friends |
| 303 | |
| 304 | ## Implementing the Server - New Friends |
| 305 | Say hello to `/ei/save_backup` and `/ei/get_periodicals`. We can infer from the name, that save_backup would involve a Backup message |
| 306 | and get_periodicals would involve a GetPeriodicalsRequest, both of which are defined fully in the proto spec file. |
| 307 | |
| 308 | Both of these are clogging up the flask application log periodically, we should check out what they are so we can have a sane log again. |
| 309 | |
| 310 | ``` |
| 311 | @app.route("/ei/<path:subpath>", methods=["POST"]) |
| 312 | def ei_routes(subpath): |
| 313 | print("HTTP POST /ei/" + subpath) |
| 314 | if subpath == "first_contact": |
| 315 | # Create the protobuf object so we can load data from the b64 payload |
| 316 | FirstContact = EIProto.EggIncFirstContactRequest() |
| 317 | FirstContact.ParseFromString(base64.b64decode(form["data"])) |
| 318 | print("We got a first contact hello from user " + FirstContact.user_id) |
| 319 | # Lets respond with a FirstContactResponse |
| 320 | FirstContactResp = EIProto.EggIncFirstContactResponse() |
| 321 | # This takes only one optional argument - a Backup object - but we have no account |
| 322 | # system yet, so we will opt out of sending that for now. |
| 323 | # --- |
| 324 | # We send the payload back as a base64 string - the same way we retrieved it. |
| 325 | return base64.b64encode(FirstContactResp.SerializeToString()) |
| 326 | elif subpath == "save_backup": |
| 327 | # NOTE: This took me way longer to realize than it should have, but the base64 |
| 328 | # payload you receive from client is broken due to some Android bug, where it |
| 329 | # substitutes "+" symbols with a " " whitespace. |
| 330 | # I don't want you to waste half hour to hours figuring out why you're getting |
| 331 | # corrupted data, so you're welcome. |
| 332 | Backup = EIProto.Backup() |
| 333 | Backup.ParseFromString(base64.b64decode(form["data"].replace(" ", "+")) |
| 334 | print(Backup) |
| 335 | elif subpath == "get_periodicals": |
| 336 | Periodicals = EIProto.GetPeriodicalsRequest() |
| 337 | Periodicals.ParseFromString(base64.b64decode(form["data"]) |
| 338 | print(Periodicals) |
| 339 | else: |
| 340 | print(request.form) |
| 341 | return "" |
| 342 | ``` |
| 343 | |
| 344 | We should now see what these payloads actually contain when deserialized (for your reading experience, I advise you to rather |
| 345 | try this out yourself - the Backup structure is VERY large). |
| 346 | |
| 347 | Upon these payloads reaching the server, we see that a very much populated Backup message makes its way through and a relatively |
| 348 | thin payload of PeriodicalsRequest comes through, which isn't very useful by itself. |
| 349 | |
| 350 | Now, this is why the game developer ended up creating forms of anticheat in future versions of this app - the Backup message contains |
| 351 | your entire game state, which is often sent as a way to save your progress to cloud, but there is no actual sanity checking in place |
| 352 | to ensure you're not progressing way too fast. Personally, I am of the mind that anticheat should be done on the server-side, not |
| 353 | on the client-side, but I digress. We can use this to prove a very obvious vulnerability when using trust-client-always architecture. |
| 354 | |
| 355 | The game has an In App Purchase for "Pro Permit", which allows you to build more Silos, which in turn allow you to get offline |
| 356 | rewards for a longer period of time. If we look at protobuf definition file, you can see under Backup.game, a field called "permit_level", |
| 357 | which by default is zero. Lets try to change that and present a modified Backup the next time user opens the game. |
| 358 | |
| 359 | |
| 360 | ``` |
| 361 | cache = {} |
| 362 | |
| 363 | @app.route("/ei/<path:subpath>", methods=["POST"]) |
| 364 | def ei_routes(subpath): |
| 365 | print("HTTP POST /ei/" + subpath) |
| 366 | if subpath == "first_contact": |
| 367 | # Create the protobuf object so we can load data from the b64 payload |
| 368 | FirstContact = EIProto.EggIncFirstContactRequest() |
| 369 | FirstContact.ParseFromString(base64.b64decode(form["data"])) |
| 370 | print("We got a first contact hello from user " + FirstContact.user_id) |
| 371 | # Lets respond with a FirstContactResponse |
| 372 | FirstContactResp = EIProto.EggIncFirstContactResponse() |
| 373 | if FirstContact.user_id in cache: |
| 374 | FirstContactResp.backup.CopyFrom(cache[FirstContact.user_id]) |
| 375 | del cache[FirstContact.user_id] |
| 376 | return base64.b64encode(FirstContactResp.SerializeToString()) |
| 377 | elif subpath == "save_backup": |
| 378 | # NOTE: This took me way longer to realize than it should have, but the base64 |
| 379 | # payload you receive from client is broken due to some Android bug, where it |
| 380 | # substitutes "+" symbols with a " " whitespace. |
| 381 | # I don't want you to waste half hour to hours figuring out why you're getting |
| 382 | # corrupted data, so you're welcome. |
| 383 | Backup = EIProto.Backup() |
| 384 | Backup.ParseFromString(base64.b64decode(form["data"].replace(" ", "+")) |
| 385 | if Backup.game.permit_level == 0: |
| 386 | print("Saved a modified Backup for next game load") |
| 387 | # Modify the permit level, force offer the backup |
| 388 | Backup.game.permit_level = 1 |
| 389 | Backup.force_offer_backup = True |
| 390 | Backup.force_backup = True |
| 391 | cache[Backup.user_id] = Backup |
| 392 | elif subpath == "get_periodicals": |
| 393 | Periodicals = EIProto.GetPeriodicalsRequest() |
| 394 | Periodicals.ParseFromString(base64.b64decode(form["data"]) |
| 395 | print(Periodicals) |
| 396 | else: |
| 397 | print(request.form) |
| 398 | return "" |
| 399 | ``` |
| 400 | |
| 401 | Lets load up the game. Nothing interesting seems to be happening yet - lets wait until we see the "Saved a modified Backup for next game load" message |
| 402 | show up in the server console. Once this shows up, restart the game - you are presented with a popup that you are offered to load a |
| 403 | Backup from server. Let's accept that. |
| 404 | |
| 405 | Now click on your silos, you have the Pro Permit for free. |
| 406 | |
| 407 | Now, it goes without saying, I do not condone piracy - the future versions of this game are very much guarded against this, rightfully so. |
| 408 | If you attempt this in actual game servers, this is considered fraud and IS detectable by the developer (every IAP has a receipt, logically!). |
| 409 | |
| 410 | This version of the game is defunct as the protocol has changed quite a bit in the years since this version and additional anticheat |
| 411 | measures have been added since. You cannot transfer this status (or even purchase what you just did) from this game version to the next. |
| 412 | |
| 413 | ### Onto the PeriodicalsRequest |
| 414 | This one is a bit more fun to delve into blindly - the proto spec wont help you much here. We will need to use our trusty RE tools again and delve into |
| 415 | the game disassembly again. |
| 416 | |
| 417 | By public knowledge, we know there are server events for "Epic Research Sale", "Research Sale", "Drone Bonus" and "Prestige Boost". We can use this information to try |
| 418 | and look at any potential leads in strings. Drone seems like a good canditate, lets look into that. |
| 419 | ``` |
| 420 | drone_fans2 |
| 421 | drone_crash |
| 422 | drone_enemy |
| 423 | drone_hunter |
| 424 | r_icon_drone_rewards |
| 425 | b_icon_drone_boost |
| 426 | drone_touch |
| 427 | ei_drone_lights_green |
| 428 | ei_drone_lights_red |
| 429 | ei_drone_package |
| 430 | ei_drone_propeller |
| 431 | drone-boost |
| 432 | GENEROUS DRONES |
| 433 | ``` |
| 434 | This looks promising, right off the bat, first strings I'd check here are `r_icon_drone_rewards`, `b_icon_drone_boost`, `drone-boost` and `GENEROUS DRONES`. |
| 435 | |
| 436 | I inspected all 4 of them, and when I got to the final 2, I found the enum string translations used for event IDs - here they are extracted for game version 1.12.13 |
| 437 | ``` |
| 438 | piggy-boost (Rate piggy fills is increased.) |
| 439 | piggy-cap-boost (UNLIMITED PIGGY;Gains are retained when event ends.) |
| 440 | prestige-boost (PRESTIGE BOOST;Collect more soul eggs on prestige, you must prestige to take advantage of this event.) |
| 441 | earnings-boost (CASH BOOST;Regular earnings are increased.) |
| 442 | gift-boost (GENEROUS GIFTS;Boost applies to random gifts and video gifts.) |
| 443 | drone-boost (GENEROUS DRONES;Drones will produce larger rewards.) |
| 444 | epic-research-sale (EPIC RESEARCH SALE;Only applies to Epic Research.) |
| 445 | vehicle-sale (VEHICLE SALE;Applies to all vehicles.) |
| 446 | boost-sale (BOOST SALE;Applies to the gold price of boosts.) |
| 447 | boost-duration (BOOST TIME+;Boosts last longer, you must start a boost during the event.) |
| 448 | ``` |
| 449 | I recall there being a few more boosts, but this is useful for getting started with compositing PeriodicalsResponse with an active running event. |
| 450 | |
| 451 | ### Putting together the response |
| 452 | We have the enum, we have the names, descriptions, lets try to create a sample server event when the client enqueries about current server periodical events. |
| 453 | ``` |
| 454 | elif subpath == "get_periodicals": |
| 455 | # We don't actually need the information client sends us, |
| 456 | # we aren't verifying any stats about client in our server. |
| 457 | CurrentPeriodicals = EIProto.PeriodicalsResponse() |
| 458 | # In order to add items to a repeatable field in protobuf structure, |
| 459 | # we need to call .add() method on it |
| 460 | event = CurrentPeriodicals.events.events.add() |
| 461 | # Refer to ei.proto - we are filling fields for EggIncEvent structure here. |
| 462 | event.type = "drone-boost" |
| 463 | event.multiplier = 5.00 |
| 464 | event.subtitle = "Drones will produce larger rewards." |
| 465 | event.identifier = "GENEROUS DRONES" |
| 466 | event.seconds_remaining = 300.0 |
| 467 | # Lets make it respond with a 5 minute event (this will re-arm itself when client calls |
| 468 | # for get_periodicals again every 6 minutes) |
| 469 | return base64.b64encode(CurrentPeriodicals.SerializeToString()) |
| 470 | ``` |
| 471 | |
| 472 | Launch the server and observe as the client periodically calls this endpoint again, it will now receive a 5 minute 5x Drone Rewards boost on the game. |
| 473 | |
| 474 | ## Created the Server - What now? |
| 475 | We have now created a very basic server, which appropriately responds to a first contact, misuses the game backup feature to prove a point about weaknesses of |
| 476 | trusting client in server. We also created a very basic server event, which always rearms itself to never expire. |
| 477 | |
| 478 | What do we do next? |
| 479 | |
| 480 | At this point, we can start dog-fooding the project. Lets start with whatever ball game throws at us as we progress. |
| 481 | |
| 482 | ### Contracts |
| 483 | As we progress the game and start performing prestiges, we unlock a feature called "Contracts" - but disaster strikes as we don't have any contracts we could |
| 484 | accept. So far we still see our good friends `/ei/get_periodicals` and `/ei/save_backup` hammering the server at regular intervals. |
| 485 | |
| 486 | When we created the periodicals response payload, you might have noticed in the protobuf message an optional field called `ContractsResponse contracts`. Lets see |
| 487 | what this ContractsResponse message contains. |
| 488 | |
| 489 | ``` |
| 490 | message ContractsResponse { |
| 491 | repeated Contract contracts = 1; |
| 492 | optional string warning_message = 4; |
| 493 | optional double server_time = 2; |
| 494 | optional uint32 max_eop = 3 [default = 1000]; |
| 495 | } |
| 496 | ``` |
| 497 | |
| 498 | Notice there being an array of Contract messages right off the bat - lets find its message structure next: |
| 499 | |
| 500 | ``` |
| 501 | message Contract { |
| 502 | optional string identifier = 1; |
| 503 | optional string name = 9; |
| 504 | optional string description = 10; |
| 505 | optional Egg egg = 2; |
| 506 | |
| 507 | repeated Goal goals = 3; |
| 508 | message Goal { |
| 509 | optional GoalType type = 1; |
| 510 | optional double target_amount = 2; |
| 511 | optional RewardType reward_type = 3; |
| 512 | optional string reward_sub_type = 4; |
| 513 | optional double reward_amount = 5; |
| 514 | optional double target_soul_eggs = 6; |
| 515 | } |
| 516 | |
| 517 | repeated GoalSet goal_sets = 16; |
| 518 | message GoalSet { |
| 519 | repeated Goal goals = 1; |
| 520 | } |
| 521 | |
| 522 | optional bool coop_allowed = 4; |
| 523 | optional uint32 max_coop_size = 5; |
| 524 | optional uint32 max_boosts = 12; |
| 525 | optional double minutes_per_token = 15 [default = 60]; |
| 526 | optional double expiration_time = 6; |
| 527 | optional double length_seconds = 7; |
| 528 | optional double max_soul_eggs = 13; |
| 529 | optional uint32 min_client_version = 14; |
| 530 | optional bool debug = 11; |
| 531 | } |
| 532 | ``` |
| 533 | |
| 534 | We will need to do a bit of reading. Fortunately, the game has a community wiki, lets look into how contracts should work. I took an |
| 535 | [older revision of Contracts wiki page from 2021](https://egg-inc.fandom.com/wiki/Contracts?oldid=13015) and did some slight research. |
| 536 | |
| 537 | From what I gather, at one point, there was only one set of contract rewards, shared between everyone - then they created a system where |
| 538 | beginners will get easier contract goals while more advanced players get harder contract goals. |
| 539 | |
| 540 | We can put two-and-two together here and infer that `repeated Goal goals` is the legacy contract system - where everyone was on equal footing |
| 541 | and `repeated GoalSet goal_sets` is the *new* goal system that is split into Standard and Elite. |
| 542 | |
| 543 | We also learn that in future game versions, they completely reworked how contracts work *yet* again into a grading "bracket" system. Fortunately, |
| 544 | we do not have to worry about that in our current target revision. |
| 545 | |
| 546 | Now to get the ball rolling, there is conveniently a starting point set ahead for us already. The developer of game intended to ease new players into |
| 547 | contracts by creating a simple & easy contract called [Your First Contract](https://egg-inc.fandom.com/wiki/Contracts/Your_First_Contract?oldid=13547). |
| 548 | |
| 549 | This page tells us all the information we need to compose our first Contract, so lets try to make one. |
| 550 | |
| 551 | ``` |
| 552 | elif subpath == "get_periodicals": |
| 553 | # We don't actually need the information client sends us, |
| 554 | # we aren't verifying any stats about client in our server. |
| 555 | CurrentPeriodicals = EIProto.PeriodicalsResponse() |
| 556 | # [...] |
| 557 | Contract = CurrentPeriodicals.contracts.contracts.add() |
| 558 | Contract.identifier = "first-contract" |
| 559 | Contract.name = "Your First Contract" |
| 560 | Contract.description = "We heard you are open to contract work! Help fill this order from the local pharmacy!" |
| 561 | Contract.egg = EIProto.Egg.MEDICAL |
| 562 | Contract.coop_allowed = False |
| 563 | Contract.minutes_per_token = 5 |
| 564 | # Lets set expiry time to always be 3 days into future |
| 565 | Contract.expiration_time = time.time() + (3600.0 * 72.0) |
| 566 | Contract.length_seconds = 3600.0 * 4.0 |
| 567 | # The wiki mentions that you cannot get this contract after you reach 5000 Soul Eggs |
| 568 | Contract.max_soul_eggs = 5000.0 |
| 569 | # We should have the basic metadata set now, lets create the goalsets. |
| 570 | FirstSet = Contract.goal_sets.add() |
| 571 | Goal = FirstSet.goals.add() |
| 572 | # There is only one type of goal in this verison |
| 573 | Goal.type = EIProto.GoalType.EGGS_LAID |
| 574 | Goal.target_amount = 100000.0 |
| 575 | Goal.reward_type = EIProto.RewardType.GOLD |
| 576 | Goal.reward_amount = 192 |
| 577 | Goal = FirstSet.goals.add() |
| 578 | Goal.type = EIProto.GoalType.EGGS_LAID |
| 579 | Goal.target_amount = 500000000.0 |
| 580 | Goal.reward_type = EIProto.RewardType.PIGGY_FILL |
| 581 | Goal.reward_amount = 10000 |
| 582 | # Lets now add the Elite table, we can pretty much copy-paste the above here. |
| 583 | SecondSet = Contract.goal_sets.add() |
| 584 | Goal = SecondSet.goals.add() |
| 585 | Goal.type = EIProto.GoalType.EGGS_LAID |
| 586 | Goal.target_amount = 100000.0 |
| 587 | Goal.reward_type = EIProto.RewardType.GOLD |
| 588 | Goal.reward_amount = 500 |
| 589 | Goal = SecondSet.goals.add() |
| 590 | Goal.type = EIProto.GoalType.EGGS_LAID |
| 591 | Goal.target_amount = 500000000.0 |
| 592 | Goal.reward_type = EIProto.RewardType.PIGGY_FILL |
| 593 | Goal.reward_amount = 10000 |
| 594 | return base64.b64encode(CurrentPeriodicals.SerializeToString()) |
| 595 | ``` |
| 596 | |
| 597 | Lets try that out in-game now - after waiting for a minute, we see our contract pop up, but I immediately noticed one thing amiss. |
| 598 | The contract goals are swapped! I am getting Elite contract rewards for a Standard contract. |
| 599 | |
| 600 | This piece of information now tells us that the first entry in GoalSets refers to Elite rewards and the second entry in GoalSets to Standard rewards. |
| 601 | After swapping the sets around, we now see a contract with the corrected rewards. |
| 602 | |
| 603 | I playtested it a bit and the contract worked as expected. |
| 604 | |
| 605 | Now, the above code could be a lot neater. For your homework, if you're not skipping to the public source release in the end, you should try to create |
| 606 | a contract database and try scheduling them like the game originally did - a "Leggacy" contract every Friday and regular contracts showing up every 1-2 weeks |
| 607 | for roughly 2 weeks. |
| 608 | |
| 609 | ## Rootless SSL Unpinning + Endpoint URL patching |
| 610 | Let's make the app not require a VPN or root privileges - let's make user CAs work and the endpoint URL something we control on the public net. |
| 611 | Start off by pulling the following repository |
| 612 | ``` |
| 613 | git clone https://github.com/ilya-kozyr/android-ssl-pinning-bypass.git |
| 614 | python3 -m venv .venv |
| 615 | source .venv/bin/activate |
| 616 | pip install -r requirements.txt |
| 617 | cp /path/to/your/apk . |
| 618 | python3 apk-rebuild.py egginc.apk --pause |
| 619 | ``` |
| 620 | |
| 621 | **NOTE!** IF you do not intend to patch the API endpoint and just want to proceed with AdAway redirecting traffic, you can stop here and press ENTER! |
| 622 | Proceed only if you own a domain in your control (that is equal or less in length to www.auxbrain.com) and want to use the app without a VPN/redirection. |
| 623 | |
| 624 | Open a new terminal window, the script will wait for us to perform modifications, enter the created folder `egginc.apk-decompiled` and `lib`. |
| 625 | |
| 626 | We have two folders here now, `arm64-v8a` and `armeabi-v7a`, just as we saw when we pulled the .so file out of the apk earlier. Let's tackle |
| 627 | the 64-bit build first. |
| 628 | |
| 629 | For arm64 build it was really simple to perform bytepatch on the said endpoint. We already know it's supposed to look as `G?www.auxbrain.com` - let's probe the .so library a bit. |
| 630 | ``` |
| 631 | $> hexdump -C libegginc.so | grep "ww.auxbrain.co" -A2 -B2 |
| 632 | 00b02b40 cd cc 4c 3f 00 00 00 00 00 00 00 00 00 00 80 3f |..L?...........?| |
| 633 | 00b02b50 00 00 00 00 00 00 00 00 00 00 00 00 14 ae 47 3f |..............G?| |
| 634 | 00b02b60 77 77 77 2e 61 75 78 62 72 61 69 6e 2e 63 6f 6d |www.auxbrain.com| |
| 635 | 00b02b70 00 48 54 54 50 20 52 45 51 3a 20 25 64 00 64 61 |.HTTP REQ: %d.da| |
| 636 | 00b02b80 74 61 3d 00 65 69 2f 66 69 72 73 74 5f 63 6f 6e |ta=.ei/first_con |
| 637 | ``` |
| 638 | |
| 639 | We seem to have nothing blocking our way, let's create hex representations of `G?www.auxbrain.com` and a target domain of equal length, for example `G?eggs.based.quest`. |
| 640 | |
| 641 | (Note: You can choose a shorter name as well, if you null-terminate the extra bytes as padding) |
| 642 | ``` |
| 643 | $> echo "G?www.auxbrain.com" | hexdump -ve '1/1 "%.2X"' |
| 644 | 473F7777772E617578627261696E2E636F6D0A |
| 645 | $> echo "G?eggs.based.quest" | hexdump -ve '1/1 "%.2X"' |
| 646 | 473F656767732E62617365642E71756573740A |
| 647 | ``` |
| 648 | |
| 649 | Remove the trailing `0A` from end of both hex strings and now proceed as follows: |
| 650 | ``` |
| 651 | # Place the source in first bracket of sed and the new URL at second bracket. |
| 652 | hexdump -ve '1/1 "%.2X"' libegginc.so | sed "s/473F7777772E617578627261696E2E636F6D/473F656767732E62617365642E7175657374/g" | xxd -r -p > patched.so |
| 653 | ``` |
| 654 | |
| 655 | Huzzah! We now have a patched linked-library for the arm64 build. Let's also patch the 32-bit version. |
| 656 | ``` |
| 657 | $> hexdump -C libegginc.so | grep "ww.auxbrain.co" -A2 -B2 |
| 658 | 0087b770 69 67 68 5f 74 6f 6f 5f 6d 61 6e 79 5f 70 78 00 |igh_too_many_px.| |
| 659 | 0087b780 74 61 62 6c 65 74 5f 68 64 70 69 00 00 00 00 00 |tablet_hdpi.....| |
| 660 | 0087b790 77 77 77 2e 61 75 78 62 72 61 69 6e 2e 63 6f 6d |www.auxbrain.com| |
| 661 | 0087b7a0 00 00 00 00 00 00 00 00 65 69 2f 66 69 72 73 74 |........ei/first| |
| 662 | 0087b7b0 5f 63 6f 6e 74 61 63 74 00 00 00 00 00 00 00 00 |_contact........| |
| 663 | ``` |
| 664 | This one lacks the `G?` prefix on API endpoint, but we still have null terminators we can rely on. Let's replace the `473F` from our previous strings with `0000`. |
| 665 | ``` |
| 666 | # Place the source in first bracket of sed and the new URL at second bracket. |
| 667 | hexdump -ve '1/1 "%.2X"' libegginc.so | sed "s/00007777772E617578627261696E2E636F6D/0000656767732E62617365642E7175657374/g" | xxd -r -p > patched.so |
| 668 | ``` |
| 669 | |
| 670 | Replace both of the libegginc.so files with the patched.so files. Move back to main terminal window and press ENTER. |
| 671 | |
| 672 | We now have a patched and debug signed apk for the game that isn't SSL pinned and contains a custom API endpoint we control without a VPN. |
| 673 | |
| 674 | ## Conclusion so far |
| 675 | We have created a (rather ugly looking) server emulator for the game. It functions, but it needs a lot of work still before we can call it ready. |
| 676 | If you have followed this far, give yourself pat on the back - if you actually tried to run this code, give yourself an extra pat on the back. |
| 677 | |
| 678 | Before I give you the public source to the project, you might want to try your hand at creating a few more things. |
| 679 | - "Cloud" save, present a Backup to any new device that just started playing. |
| 680 | - Contracts Database and scheduler |
| 681 | - Server Event scheduler |
| 682 | |
| 683 | I apologize if my method of documenting this has been messy, but that's also part of the chaos of reverse engineering, you are constantly learning new things |
| 684 | about the project you are currently doing - refactoring becomes an essential part once you have documented the protocol to a comfortable degree. |
| 685 | |
| 686 | I won't give any promises for a part 2 any time soon, but I will be trying to make this feature complete, so without further ado, here are the git repository links: |
| 687 | [github.com](https://github.com/cernodile/reEgg), [git.based.quest](https://git.based.quest/?p=reEgg.git;a=tree;h=refs/heads/master;hb=refs/heads/master). |
| 688 | |
| 689 | Thank you for reading and making it all the way to the end, |
| 690 | - Cernodile |