Oops. SSL was unnecessary in this example.
[web-hugo.git] / content / posts / reverse-engineering-a-mobile-app-protobuf-api.md
CommitLineData
2a5bb5a5
JM
1---
2title: "Reverse Engineering a Mobile App Protobuf API"
3date: 2024-05-11T12:00:00+03:00
4description: "In recent times more than ever, live service and overuse of APIs soon-to-be-stale is only increasing. This causes a lot of once-written software to become either unusable or handicapped in many regards. One way to fight this is to learn reverse engineering for sake of digital preservation. In this blog, I take you through a journey of reverse engineering a simple mobile game's protobuf API."
5tags: ['tutorial', 'reverse-engineering', 'opensource']
6type: blog
7draft: false
8---
9
10# Why
11Why not? Digital preservation is important, even if you don't care for a specific program.
12This is also a good way to get started with protocol reverse engineering due to way protobuf
13is often left behind in source format within client applications.
14
15## The target
16In this series of blogposts, I will be using a mobile game "Egg, Inc." as the target for
17demonstration. It's a simple time killer app that got me through boring long waits when I was still at school.
18
1e75f355
JM
19Egg, Inc. is a basic incremental "idler" game where your goal is to take over the world food supply with ever-increasing supply of eggs,
20if you have ever played Cookie Clicker, you know the premise of something like that. You have to unlock denser and denser eggs - the game
21is also designed around the fact that you can do certain online-tied activites such as Contracts to unlock more Soul Eggs (prestige boost) and
22"Eggs of Prophecy" which increase potency of your Soul Eggs.
23
24It's rather simple game with a very minimal API, making it perfect for learning. You may not like the game, but that's beside the point.
25The simplicity of our target matters here.
2a5bb5a5
JM
26
27## The existing works
1e75f355
JM
28In some cases, you will find previous works on the target you pick. In my case, some clever people have created
29[scripts to extract .proto file out of the app.](https://github.com/DavidArthurCole/EggIncProtoExtractor)
2a5bb5a5
JM
30I advise you to check it out if you wish to get a better understanding of how you would go about retrieving the
31API spec .proto file for your target.
32
33Further there are a few dedicated individuals in the game's community who have created numerous tools and historical databases.
34
35For this blog purposes, we will assume the game server is shut down (as in we cannot query from the live API) and our goal is
36to make a semi-functional selfhosted gameserver for our own needs, assuming we are the only one on said server.
37
1e75f355
JM
38## How to source builds of a game
39There are two methods of sourcing the apk file here - one method is if you already have the app installed, install something like ZArchiver
40and extract it from /data/app/ - identifying the app by its icon. From there you will find `base.apk` which is enough for most apps.
41
42Alternatively, if the app is still available on Google Play, you can use an app like Aurora Store to go to the store detail page, select
43"Manual Download" and enter a known Build ID.
44
2a5bb5a5
JM
45## Getting Started
46Thanks to the previously mentioned script, it's easy to get started - find the APK, extract protobuf spec file, convert it with
47protoc and we're done there. One small problem - due to cheaters, latest version of the game includes "AuthenticatedMessage" structure,
48which contains a salted sha256sum of the payload message.
49
50At this point, after a bit of internal dilemma, I decided to not further the problem while service is still live for people playing and did the
51more morally sound decision of picking a version prior to these integrity checks. We can crack that another day as all the needed information
52is retained in the app itself.
53
1e75f355 54Going forward with this, we are targetting game version 1.12.13 (Build ID 111121 - use that in Aurora Store).
2a5bb5a5
JM
55
56With all that out of the way, lets get into actual commands used here:
57```
58git clone https://github.com/DavidArthurCole/EggIncProtoExtractor.git
59cd EggIncProtoExtractor
60./apkextract.sh com.auxbrain.egginc_1.12.13.apk
61# We should have a new folder "protos" now with resulting files
62cd protos
63# There should be a file called ei.proto - that's our protobuf spec file
64# At this point, we can use the protoc utility which can convert the specfile
65# to interfaces in C++, C#, Java, Kotlin, Objective-C, PHP, Python and Ruby with
66# additional plugin support for Dart and Go.
67# To make this easier to understand, we will use Python in this demonstration
68protoc -I=. --python_out=. ./ei.proto
69# Success! We now have a "ei_pb2.py" file which can be directly imported to Python programs
70```
71
72With the protobuf interface in Python created, we can now proceed with creating the API emulator - but there's a slight problem.
73What URL? What endpoints? How do we find this out? Simple answer, disassembling the game. Get your RE tool of choice, I will be
74using [Ghidra](https://ghidra-sre.org/) myself.
75
76(Note: You can also just try to find this out using tools such as WireShark)
77
78The game contains a linked-library written in C++, which you can find inside the .apk `lib` folder, named as `libegginc.so`.
79This is perfect for our use-case, Ghidra is going to slice through this like butter. Import the file to your RE tool of choice
80and let it perform some analysis on it, have a cup of tea or coffee as this is going to take a hot minute.
81
82Once that's done, we are going to start by looking at the defined strings - try our luck there. Search for any debug prints left
83behind or maybe some clues. I started by searching for `http`, which lead me to following string `"HTTP REQ: %d"`, seems promising.
84When I jumped to it, I saw an exactly adjacent string to it which could give more clues:
85```
86 s_www.auxbrain.com_00c02b60 XREF[0,1]: FUN_00518ab8:00518b38(R)
87 00c02b5e 47 3f 77 ds "G?www.auxbrain.com"
88 77 77 2e
89 61 75 78
90 s_HTTP_REQ:_%d_00c02b71 XREF[1]: makeRequestInternal:0067bbd4(*)
91 00c02b71 48 54 54 ds "HTTP REQ: %d"
92 50 20 52
93 45 51 3a
94```
95Interesting, `www.auxbrain.com`. If we jump to its XREF, we get a garbled function, but what it seems to be doing is setting up
96certain global values.
97
be8f6a91
JM
98## The smoke-test
99
2a5bb5a5 100So we have a potential API endpoint, let's put it to the test. We're not going to recompile anything yet or do any byte-patching,
37ff2de8
JM
101let's try a quick smoke-test.
102
103**UPDATE 15/05/2024: It turns out this old app version uses HTTP only, instead of HTTPS. You will only need to perform the AdAway instruction here.
104As this information is still vastly useful for reverse engineering most apps, I will be leaving this section intact.**
105
106Ensure your phone is rooted and you have a variant of Xposed Framework installed (I used LSPosed).
1e75f355 107We will need to unarm the SSL pinning present in most apps, including this one, I used [io.github.tehcneko.sslunpinning](https://github.com/Xposed-Modules-Repo/io.github.tehcneko.sslunpinning) module.
9656149b 108(NOTE: Users without root might want to skip to end of article where I showcase unpinning the app manually)
2a5bb5a5
JM
109
110Next, install [AdAway app from F-Droid](https://f-droid.org/packages/org.adaway/) so we can setup a redirection on any network we are on.
111Inside AdAway, add a redirection rule for the address we just found and point it to an IP address in your LAN that will run the API server.
112
113Generate a self-signed certificate authority and a certificate signed by it and run a webserver with both HTTP and HTTPS on the API server machine.
be8f6a91
JM
114```
115# Create an ext file containing the Subject Alternative Name (SAN)
9656149b
JM
116# DNS.1 should correspond to the API endpoint of the app.
117# NOTE! If you are changing the API endpoint to a public domain, you can just use a public cert, no need for any of this.
be8f6a91
JM
118cat > auxbrain.ext << EOF
119authorityKeyIdentifier=keyid,issuer
120basicConstraints=CA:FALSE
121keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
122subjectAltName = @alt_names
123
124[alt_names]
125DNS.1 = www.auxbrain.com
126EOF
127
128# Create your own Certificate Authority
129openssl genrsa -des3 -out myCA.key 2048
130openssl req -x509 -new -nodes -key myCA.key -sha256 -days 1825 -out myCA.pem
131# Create a CSR and lets have the new CA sign it
3bcb1df5 132openssl genrsa -out auxbrain.key 2048
be8f6a91 133openssl req -new -key auxbrain.key -out auxbrain.csr -nodes
f055cad6
JM
134openssl x509 -req -in auxbrain.csr -CA myCA.pem -CAkey myCA.key -CAcreateserial -out auxbrain.crt -days 825 -sha256 -extfile auxbrain.ext
135cat auxbrain.crt myCA.pem > auxbrain.pem
596663d8
JM
136# You now have:
137# myCA.pem - the public certificate of your root CA
138# auxbrain.key - the private key for your webserver
139# auxbrain.pem - the public cert for your webserver.
be8f6a91
JM
140```
141
142Use the generated `auxbrain.pem` and `auxbrain.key` files for your webserver SSL/TLS configuration. For nginx, append following values to your server directive:
143```
144listen 443 ssl;
145ssl_certificate /path/to/auxbrain.pem;
146ssl_certificate_key /path/to/auxbrain.key;
147ssl_session_cache shared:SSL:1m;
148ssl_session_timeout 5m;
149ssl_ciphers HIGH:!aNULL:!MD5;
150ssl_prefer_server_ciphers on;
151```
152
153Import the self-signed CA (myCA.pem) to your phone's truststore (Check under your phone's Security/Encryption settings). Once all of that is done, run the app for first time.
2a5bb5a5
JM
154
155```
156192.168.1.212 - - [...] "POST /ei/first_contact HTTP/1.1" 404 0 "-"
157```
158
159Bingo. We have contact and we have an API endpoint. Searching for "ei/" in the strings reveals a extensive list of API endpoints, we now have something
160to go off from. We have everything we need to start creating the server.
161
162## Implementing the Server - Getting first contact
163Next up, we create a new project - as we generated the protobuf definitions for Python, we will proceed accordingly.
164If you are following along, get respective packages for your operating system to create python venvs.
165As the protobufs are being sent over HTTP, we will be serving our application over flask which is being reverse proxied by nginx.
166
167```
168# Lets stage the environment
169mkdir apiserver
170cd apiserver
171python -m venv .venv
172source .venv/bin/activate
173touch app.py
174cp ~/EggIncProtoExtractor/protos/ei.proto .
175
176# Get some dependeices
177pip install protobuf
178pip install flask
179```
180
181We now have the project set up for reading protobuf definitions and a framework to listen for HTTP and routes sent to it.
182Let's create an actual listener application, open app.py with your favourite IDE or text editor.
183
184```
185import ei_pb2 as EIProto
186from flask import Flask
187from flask import request
188
189@app.route("/ei/<path:subpath>", methods=["POST"])
190def ei_routes(subpath):
191 print("HTTP POST /ei/" + subpath)
192 print(request.headers)
193 return ""
194```
195
196This should get the ball rolling, we will see whatever call comes in and we can see what the payload of each request contains.
197At this point you should setup the reverse proxy, override your nginx / directive with:
198```
199location / {
200 proxy_pass http://127.0.0.1:5000;
201}
202```
203
204Reload your nginx and start the flask application you just created with `flask run`.
205
206Run the app again and have it phone home and see what it contains.
207```
208HTTP POST /ei/first_contact
209Host: 127.0.0.1:5000
210Connection: close
211Content-Length: 37
212Content-Type: application/x-www-form-urlencoded
213User-Agent: Dalvik/2.1.0 (Linux; U; Android 13; M2012K11AG Build/TQ3A.230901.001)
214Accept-Encoding: gzip
215```
216
217We can see there's a form payload attached to this request, let's modify our app route a bit:
218```
219@app.route("/ei/<path:subpath>", methods=["POST"])
220def ei_routes(subpath):
221 print("HTTP POST /ei/" + subpath)
222 print(request.form)
223 return ""
224```
225
226Now if we run the modified flask application again, we see following output on the first_contact endpoint.
227```
228HTTP POST /ei/first_contact
229ImmutableMultiDict([('data', 'ChAzNTVlNDZlOTA4OWQxZTRjEAAYAg==')])
230```
231
232We have a base64-encoded protobuf binary data - which isn't terribly useful for reading plain-text, since protobuf *is* a binary
233format, so we will need to figure out what protobuf message this payload belongs to.
234
235Remember that ei.proto file alongside the ei_pb2.py we got earlier? Lets go back there and inspect it a bit.
236We know we just contacted something called "first_contact", maybe there is something in that file that could help us?
237
238```
239message EggIncFirstContactRequest {
240 optional string user_id = 1;
241 optional uint32 client_version = 2;
242 optional Platform platform = 3;
243}
244
245message EggIncFirstContactResponse {
246 optional Backup backup = 1;
247}
248```
249
250Seems like the application is using message names in almost similar fashion to API endpoint names themselves. This will prove
251to be useful knowledge. We now know what the payload should be, lets put this to the test.
252
253Edit your app routine again
254```
255# add "import base64" to top of the file
256@app.route("/ei/<path:subpath>", methods=["POST"])
257def ei_routes(subpath):
258 print("HTTP POST /ei/" + subpath)
259 if subpath == "first_contact":
260 # Create the protobuf object so we can load data from the b64 payload
261 FirstContact = EIProto.EggIncFirstContactRequest()
262 FirstContact.ParseFromString(base64.b64decode(form["data"]))
263 print(FirstContact)
264 else:
265 print(request.form)
266 return ""
267```
268
269We should now be able to see deserialized output when we run the flask application and the mobile app, let's try it out:
270
271```
272HTTP POST /ei/first_contact
273user_id: "355e46e9089d1e4c"
274client_version: 0
275platform: DROID
276```
277
278Nice! We now know how to identify which protobuf object corresponds to which API endpoint. We can now make an educated guess
279on what would come next.
280
281Seeing how we got `EggIncFirstContactRequest` and saw an adjacent `EggIncFirstContactResponse` message in the proto file, we
282can safely assume that this is what the game is expecting from us in return.
283
284Lets modify the server a bit to account for that.
285
286```
287@app.route("/ei/<path:subpath>", methods=["POST"])
288def ei_routes(subpath):
289 print("HTTP POST /ei/" + subpath)
290 if subpath == "first_contact":
291 # Create the protobuf object so we can load data from the b64 payload
292 FirstContact = EIProto.EggIncFirstContactRequest()
293 FirstContact.ParseFromString(base64.b64decode(form["data"]))
294 print("We got a first contact hello from user " + FirstContact.user_id)
295 # Lets respond with a FirstContactResponse
296 FirstContactResp = EIProto.EggIncFirstContactResponse()
297 # This takes only one optional argument - a Backup object - but we have no account
298 # system yet, so we will opt out of sending that for now.
299 # ---
300 # We send the payload back as a base64 string - the same way we retrieved it.
301 return base64.b64encode(FirstContactResp.SerializeToString())
302 else:
303 print(request.form)
304 return ""
305```
306
307Now when we run the app again, we notice that we no longer get spammed this endpoint, but instead in its place we see a few new friends
308
309## Implementing the Server - New Friends
310Say hello to `/ei/save_backup` and `/ei/get_periodicals`. We can infer from the name, that save_backup would involve a Backup message
311and get_periodicals would involve a GetPeriodicalsRequest, both of which are defined fully in the proto spec file.
312
313Both of these are clogging up the flask application log periodically, we should check out what they are so we can have a sane log again.
314
315```
316@app.route("/ei/<path:subpath>", methods=["POST"])
317def ei_routes(subpath):
318 print("HTTP POST /ei/" + subpath)
319 if subpath == "first_contact":
320 # Create the protobuf object so we can load data from the b64 payload
321 FirstContact = EIProto.EggIncFirstContactRequest()
322 FirstContact.ParseFromString(base64.b64decode(form["data"]))
323 print("We got a first contact hello from user " + FirstContact.user_id)
324 # Lets respond with a FirstContactResponse
325 FirstContactResp = EIProto.EggIncFirstContactResponse()
326 # This takes only one optional argument - a Backup object - but we have no account
327 # system yet, so we will opt out of sending that for now.
328 # ---
329 # We send the payload back as a base64 string - the same way we retrieved it.
330 return base64.b64encode(FirstContactResp.SerializeToString())
331 elif subpath == "save_backup":
332 # NOTE: This took me way longer to realize than it should have, but the base64
333 # payload you receive from client is broken due to some Android bug, where it
334 # substitutes "+" symbols with a " " whitespace.
335 # I don't want you to waste half hour to hours figuring out why you're getting
336 # corrupted data, so you're welcome.
337 Backup = EIProto.Backup()
338 Backup.ParseFromString(base64.b64decode(form["data"].replace(" ", "+"))
339 print(Backup)
340 elif subpath == "get_periodicals":
341 Periodicals = EIProto.GetPeriodicalsRequest()
342 Periodicals.ParseFromString(base64.b64decode(form["data"])
343 print(Periodicals)
344 else:
345 print(request.form)
346 return ""
347```
348
349We should now see what these payloads actually contain when deserialized (for your reading experience, I advise you to rather
350try this out yourself - the Backup structure is VERY large).
351
352Upon these payloads reaching the server, we see that a very much populated Backup message makes its way through and a relatively
353thin payload of PeriodicalsRequest comes through, which isn't very useful by itself.
354
355Now, this is why the game developer ended up creating forms of anticheat in future versions of this app - the Backup message contains
356your entire game state, which is often sent as a way to save your progress to cloud, but there is no actual sanity checking in place
357to ensure you're not progressing way too fast. Personally, I am of the mind that anticheat should be done on the server-side, not
358on the client-side, but I digress. We can use this to prove a very obvious vulnerability when using trust-client-always architecture.
359
360The game has an In App Purchase for "Pro Permit", which allows you to build more Silos, which in turn allow you to get offline
361rewards for a longer period of time. If we look at protobuf definition file, you can see under Backup.game, a field called "permit_level",
362which by default is zero. Lets try to change that and present a modified Backup the next time user opens the game.
363
364
365```
366cache = {}
367
368@app.route("/ei/<path:subpath>", methods=["POST"])
369def ei_routes(subpath):
370 print("HTTP POST /ei/" + subpath)
371 if subpath == "first_contact":
372 # Create the protobuf object so we can load data from the b64 payload
373 FirstContact = EIProto.EggIncFirstContactRequest()
374 FirstContact.ParseFromString(base64.b64decode(form["data"]))
375 print("We got a first contact hello from user " + FirstContact.user_id)
376 # Lets respond with a FirstContactResponse
377 FirstContactResp = EIProto.EggIncFirstContactResponse()
378 if FirstContact.user_id in cache:
379 FirstContactResp.backup.CopyFrom(cache[FirstContact.user_id])
380 del cache[FirstContact.user_id]
381 return base64.b64encode(FirstContactResp.SerializeToString())
382 elif subpath == "save_backup":
383 # NOTE: This took me way longer to realize than it should have, but the base64
384 # payload you receive from client is broken due to some Android bug, where it
385 # substitutes "+" symbols with a " " whitespace.
386 # I don't want you to waste half hour to hours figuring out why you're getting
387 # corrupted data, so you're welcome.
388 Backup = EIProto.Backup()
389 Backup.ParseFromString(base64.b64decode(form["data"].replace(" ", "+"))
390 if Backup.game.permit_level == 0:
391 print("Saved a modified Backup for next game load")
392 # Modify the permit level, force offer the backup
393 Backup.game.permit_level = 1
394 Backup.force_offer_backup = True
395 Backup.force_backup = True
396 cache[Backup.user_id] = Backup
397 elif subpath == "get_periodicals":
398 Periodicals = EIProto.GetPeriodicalsRequest()
399 Periodicals.ParseFromString(base64.b64decode(form["data"])
400 print(Periodicals)
401 else:
402 print(request.form)
403 return ""
404```
405
406Lets load up the game. Nothing interesting seems to be happening yet - lets wait until we see the "Saved a modified Backup for next game load" message
407show up in the server console. Once this shows up, restart the game - you are presented with a popup that you are offered to load a
408Backup from server. Let's accept that.
409
410Now click on your silos, you have the Pro Permit for free.
411
412Now, it goes without saying, I do not condone piracy - the future versions of this game are very much guarded against this, rightfully so.
413If you attempt this in actual game servers, this is considered fraud and IS detectable by the developer (every IAP has a receipt, logically!).
414
415This version of the game is defunct as the protocol has changed quite a bit in the years since this version and additional anticheat
416measures have been added since. You cannot transfer this status (or even purchase what you just did) from this game version to the next.
417
418### Onto the PeriodicalsRequest
419This one is a bit more fun to delve into blindly - the proto spec wont help you much here. We will need to use our trusty RE tools again and delve into
420the game disassembly again.
421
422By public knowledge, we know there are server events for "Epic Research Sale", "Research Sale", "Drone Bonus" and "Prestige Boost". We can use this information to try
423and look at any potential leads in strings. Drone seems like a good canditate, lets look into that.
424```
425drone_fans2
426drone_crash
427drone_enemy
428drone_hunter
429r_icon_drone_rewards
430b_icon_drone_boost
431drone_touch
432ei_drone_lights_green
433ei_drone_lights_red
434ei_drone_package
435ei_drone_propeller
436drone-boost
437GENEROUS DRONES
438```
439This looks promising, right off the bat, first strings I'd check here are `r_icon_drone_rewards`, `b_icon_drone_boost`, `drone-boost` and `GENEROUS DRONES`.
440
1e75f355 441I inspected all 4 of them, and when I got to the final 2, I found the enum string translations used for event IDs - here they are extracted for game version 1.12.13
2a5bb5a5
JM
442```
443piggy-boost (Rate piggy fills is increased.)
444piggy-cap-boost (UNLIMITED PIGGY;Gains are retained when event ends.)
445prestige-boost (PRESTIGE BOOST;Collect more soul eggs on prestige, you must prestige to take advantage of this event.)
446earnings-boost (CASH BOOST;Regular earnings are increased.)
447gift-boost (GENEROUS GIFTS;Boost applies to random gifts and video gifts.)
448drone-boost (GENEROUS DRONES;Drones will produce larger rewards.)
449epic-research-sale (EPIC RESEARCH SALE;Only applies to Epic Research.)
450vehicle-sale (VEHICLE SALE;Applies to all vehicles.)
451boost-sale (BOOST SALE;Applies to the gold price of boosts.)
452boost-duration (BOOST TIME+;Boosts last longer, you must start a boost during the event.)
453```
454I recall there being a few more boosts, but this is useful for getting started with compositing PeriodicalsResponse with an active running event.
455
456### Putting together the response
457We have the enum, we have the names, descriptions, lets try to create a sample server event when the client enqueries about current server periodical events.
458```
459 elif subpath == "get_periodicals":
460 # We don't actually need the information client sends us,
461 # we aren't verifying any stats about client in our server.
462 CurrentPeriodicals = EIProto.PeriodicalsResponse()
463 # In order to add items to a repeatable field in protobuf structure,
464 # we need to call .add() method on it
465 event = CurrentPeriodicals.events.events.add()
466 # Refer to ei.proto - we are filling fields for EggIncEvent structure here.
467 event.type = "drone-boost"
468 event.multiplier = 5.00
469 event.subtitle = "Drones will produce larger rewards."
470 event.identifier = "GENEROUS DRONES"
471 event.seconds_remaining = 300.0
472 # Lets make it respond with a 5 minute event (this will re-arm itself when client calls
473 # for get_periodicals again every 6 minutes)
474 return base64.b64encode(CurrentPeriodicals.SerializeToString())
475```
476
477Launch the server and observe as the client periodically calls this endpoint again, it will now receive a 5 minute 5x Drone Rewards boost on the game.
478
479## Created the Server - What now?
480We have now created a very basic server, which appropriately responds to a first contact, misuses the game backup feature to prove a point about weaknesses of
481trusting client in server. We also created a very basic server event, which always rearms itself to never expire.
482
483What do we do next?
484
485At this point, we can start dog-fooding the project. Lets start with whatever ball game throws at us as we progress.
486
487### Contracts
488As we progress the game and start performing prestiges, we unlock a feature called "Contracts" - but disaster strikes as we don't have any contracts we could
1e75f355 489accept. So far we still see our good friends `/ei/get_periodicals` and `/ei/save_backup` hammering the server at regular intervals.
2a5bb5a5
JM
490
491When we created the periodicals response payload, you might have noticed in the protobuf message an optional field called `ContractsResponse contracts`. Lets see
492what this ContractsResponse message contains.
493
494```
495message ContractsResponse {
496 repeated Contract contracts = 1;
497 optional string warning_message = 4;
498 optional double server_time = 2;
499 optional uint32 max_eop = 3 [default = 1000];
500}
501```
502
503Notice there being an array of Contract messages right off the bat - lets find its message structure next:
504
505```
506message Contract {
507 optional string identifier = 1;
508 optional string name = 9;
509 optional string description = 10;
510 optional Egg egg = 2;
511
512 repeated Goal goals = 3;
513 message Goal {
514 optional GoalType type = 1;
515 optional double target_amount = 2;
516 optional RewardType reward_type = 3;
517 optional string reward_sub_type = 4;
518 optional double reward_amount = 5;
519 optional double target_soul_eggs = 6;
520 }
521
522 repeated GoalSet goal_sets = 16;
523 message GoalSet {
524 repeated Goal goals = 1;
525 }
526
527 optional bool coop_allowed = 4;
528 optional uint32 max_coop_size = 5;
529 optional uint32 max_boosts = 12;
530 optional double minutes_per_token = 15 [default = 60];
531 optional double expiration_time = 6;
532 optional double length_seconds = 7;
533 optional double max_soul_eggs = 13;
534 optional uint32 min_client_version = 14;
535 optional bool debug = 11;
536}
537```
538
539We will need to do a bit of reading. Fortunately, the game has a community wiki, lets look into how contracts should work. I took an
540[older revision of Contracts wiki page from 2021](https://egg-inc.fandom.com/wiki/Contracts?oldid=13015) and did some slight research.
541
542From what I gather, at one point, there was only one set of contract rewards, shared between everyone - then they created a system where
543beginners will get easier contract goals while more advanced players get harder contract goals.
544
545We can put two-and-two together here and infer that `repeated Goal goals` is the legacy contract system - where everyone was on equal footing
546and `repeated GoalSet goal_sets` is the *new* goal system that is split into Standard and Elite.
547
1e75f355 548We also learn that in future game versions, they completely reworked how contracts work *yet* again into a grading "bracket" system. Fortunately,
2a5bb5a5
JM
549we do not have to worry about that in our current target revision.
550
551Now to get the ball rolling, there is conveniently a starting point set ahead for us already. The developer of game intended to ease new players into
552contracts by creating a simple & easy contract called [Your First Contract](https://egg-inc.fandom.com/wiki/Contracts/Your_First_Contract?oldid=13547).
553
554This page tells us all the information we need to compose our first Contract, so lets try to make one.
555
556```
557 elif subpath == "get_periodicals":
558 # We don't actually need the information client sends us,
559 # we aren't verifying any stats about client in our server.
560 CurrentPeriodicals = EIProto.PeriodicalsResponse()
561 # [...]
562 Contract = CurrentPeriodicals.contracts.contracts.add()
563 Contract.identifier = "first-contract"
564 Contract.name = "Your First Contract"
565 Contract.description = "We heard you are open to contract work! Help fill this order from the local pharmacy!"
566 Contract.egg = EIProto.Egg.MEDICAL
567 Contract.coop_allowed = False
568 Contract.minutes_per_token = 5
569 # Lets set expiry time to always be 3 days into future
570 Contract.expiration_time = time.time() + (3600.0 * 72.0)
571 Contract.length_seconds = 3600.0 * 4.0
572 # The wiki mentions that you cannot get this contract after you reach 5000 Soul Eggs
573 Contract.max_soul_eggs = 5000.0
574 # We should have the basic metadata set now, lets create the goalsets.
575 FirstSet = Contract.goal_sets.add()
576 Goal = FirstSet.goals.add()
577 # There is only one type of goal in this verison
578 Goal.type = EIProto.GoalType.EGGS_LAID
579 Goal.target_amount = 100000.0
580 Goal.reward_type = EIProto.RewardType.GOLD
581 Goal.reward_amount = 192
582 Goal = FirstSet.goals.add()
583 Goal.type = EIProto.GoalType.EGGS_LAID
584 Goal.target_amount = 500000000.0
585 Goal.reward_type = EIProto.RewardType.PIGGY_FILL
586 Goal.reward_amount = 10000
587 # Lets now add the Elite table, we can pretty much copy-paste the above here.
588 SecondSet = Contract.goal_sets.add()
589 Goal = SecondSet.goals.add()
590 Goal.type = EIProto.GoalType.EGGS_LAID
591 Goal.target_amount = 100000.0
592 Goal.reward_type = EIProto.RewardType.GOLD
593 Goal.reward_amount = 500
594 Goal = SecondSet.goals.add()
595 Goal.type = EIProto.GoalType.EGGS_LAID
596 Goal.target_amount = 500000000.0
597 Goal.reward_type = EIProto.RewardType.PIGGY_FILL
598 Goal.reward_amount = 10000
599 return base64.b64encode(CurrentPeriodicals.SerializeToString())
600```
601
1e75f355 602Lets try that out in-game now - after waiting for a minute, we see our contract pop up, but I immediately noticed one thing amiss.
2a5bb5a5
JM
603The contract goals are swapped! I am getting Elite contract rewards for a Standard contract.
604
605This piece of information now tells us that the first entry in GoalSets refers to Elite rewards and the second entry in GoalSets to Standard rewards.
606After swapping the sets around, we now see a contract with the corrected rewards.
607
608I playtested it a bit and the contract worked as expected.
609
610Now, the above code could be a lot neater. For your homework, if you're not skipping to the public source release in the end, you should try to create
611a contract database and try scheduling them like the game originally did - a "Leggacy" contract every Friday and regular contracts showing up every 1-2 weeks
612for roughly 2 weeks.
613
1e75f355
JM
614## Rootless SSL Unpinning + Endpoint URL patching
615Let's make the app not require a VPN or root privileges - let's make user CAs work and the endpoint URL something we control on the public net.
616Start off by pulling the following repository
617```
618git clone https://github.com/ilya-kozyr/android-ssl-pinning-bypass.git
619python3 -m venv .venv
620source .venv/bin/activate
621pip install -r requirements.txt
622cp /path/to/your/apk .
623python3 apk-rebuild.py egginc.apk --pause
624```
625
9656149b
JM
626**NOTE!** IF you do not intend to patch the API endpoint and just want to proceed with AdAway redirecting traffic, you can stop here and press ENTER!
627Proceed only if you own a domain in your control (that is equal or less in length to www.auxbrain.com) and want to use the app without a VPN/redirection.
628
1e75f355
JM
629Open a new terminal window, the script will wait for us to perform modifications, enter the created folder `egginc.apk-decompiled` and `lib`.
630
631We have two folders here now, `arm64-v8a` and `armeabi-v7a`, just as we saw when we pulled the .so file out of the apk earlier. Let's tackle
632the 64-bit build first.
633
634For arm64 build it was really simple to perform bytepatch on the said endpoint. We already know it's supposed to look as `G?www.auxbrain.com` - let's probe the .so library a bit.
635```
636$> hexdump -C libegginc.so | grep "ww.auxbrain.co" -A2 -B2
63700b02b40 cd cc 4c 3f 00 00 00 00 00 00 00 00 00 00 80 3f |..L?...........?|
63800b02b50 00 00 00 00 00 00 00 00 00 00 00 00 14 ae 47 3f |..............G?|
63900b02b60 77 77 77 2e 61 75 78 62 72 61 69 6e 2e 63 6f 6d |www.auxbrain.com|
64000b02b70 00 48 54 54 50 20 52 45 51 3a 20 25 64 00 64 61 |.HTTP REQ: %d.da|
64100b02b80 74 61 3d 00 65 69 2f 66 69 72 73 74 5f 63 6f 6e |ta=.ei/first_con
642```
643
644We seem to have nothing blocking our way, let's create hex representations of `G?www.auxbrain.com` and a target domain of equal length, for example `G?eggs.based.quest`.
645
646(Note: You can choose a shorter name as well, if you null-terminate the extra bytes as padding)
647```
648$> echo "G?www.auxbrain.com" | hexdump -ve '1/1 "%.2X"'
649473F7777772E617578627261696E2E636F6D0A
650$> echo "G?eggs.based.quest" | hexdump -ve '1/1 "%.2X"'
651473F656767732E62617365642E71756573740A
652```
653
654Remove the trailing `0A` from end of both hex strings and now proceed as follows:
655```
656# Place the source in first bracket of sed and the new URL at second bracket.
657hexdump -ve '1/1 "%.2X"' libegginc.so | sed "s/473F7777772E617578627261696E2E636F6D/473F656767732E62617365642E7175657374/g" | xxd -r -p > patched.so
658```
659
660Huzzah! We now have a patched linked-library for the arm64 build. Let's also patch the 32-bit version.
661```
662$> hexdump -C libegginc.so | grep "ww.auxbrain.co" -A2 -B2
6630087b770 69 67 68 5f 74 6f 6f 5f 6d 61 6e 79 5f 70 78 00 |igh_too_many_px.|
6640087b780 74 61 62 6c 65 74 5f 68 64 70 69 00 00 00 00 00 |tablet_hdpi.....|
6650087b790 77 77 77 2e 61 75 78 62 72 61 69 6e 2e 63 6f 6d |www.auxbrain.com|
6660087b7a0 00 00 00 00 00 00 00 00 65 69 2f 66 69 72 73 74 |........ei/first|
6670087b7b0 5f 63 6f 6e 74 61 63 74 00 00 00 00 00 00 00 00 |_contact........|
668```
669This one lacks the `G?` prefix on API endpoint, but we still have null terminators we can rely on. Let's replace the `473F` from our previous strings with `0000`.
670```
671# Place the source in first bracket of sed and the new URL at second bracket.
672hexdump -ve '1/1 "%.2X"' libegginc.so | sed "s/00007777772E617578627261696E2E636F6D/0000656767732E62617365642E7175657374/g" | xxd -r -p > patched.so
673```
674
675Replace both of the libegginc.so files with the patched.so files. Move back to main terminal window and press ENTER.
676
677We now have a patched and debug signed apk for the game that isn't SSL pinned and contains a custom API endpoint we control without a VPN.
678
2a5bb5a5
JM
679## Conclusion so far
680We have created a (rather ugly looking) server emulator for the game. It functions, but it needs a lot of work still before we can call it ready.
681If you have followed this far, give yourself pat on the back - if you actually tried to run this code, give yourself an extra pat on the back.
682
683Before I give you the public source to the project, you might want to try your hand at creating a few more things.
684- "Cloud" save, present a Backup to any new device that just started playing.
685- Contracts Database and scheduler
686- Server Event scheduler
687
688I apologize if my method of documenting this has been messy, but that's also part of the chaos of reverse engineering, you are constantly learning new things
689about the project you are currently doing - refactoring becomes an essential part once you have documented the protocol to a comfortable degree.
690
691I won't give any promises for a part 2 any time soon, but I will be trying to make this feature complete, so without further ado, here are the git repository links:
692[github.com](https://github.com/cernodile/reEgg), [git.based.quest](https://git.based.quest/?p=reEgg.git;a=tree;h=refs/heads/master;hb=refs/heads/master).
693
694Thank you for reading and making it all the way to the end,
695- Cernodile