Cert must be fullchain for most webservers
[web-hugo.git] / content / posts / reverse-engineering-a-mobile-app-protobuf-api.md
CommitLineData
2a5bb5a5
JM
1---
2title: "Reverse Engineering a Mobile App Protobuf API"
3date: 2024-05-11T12:00:00+03:00
4description: "In recent times more than ever, live service and overuse of APIs soon-to-be-stale is only increasing. This causes a lot of once-written software to become either unusable or handicapped in many regards. One way to fight this is to learn reverse engineering for sake of digital preservation. In this blog, I take you through a journey of reverse engineering a simple mobile game's protobuf API."
5tags: ['tutorial', 'reverse-engineering', 'opensource']
6type: blog
7draft: false
8---
9
10# Why
11Why not? Digital preservation is important, even if you don't care for a specific program.
12This is also a good way to get started with protocol reverse engineering due to way protobuf
13is often left behind in source format within client applications.
14
15## The target
16In this series of blogposts, I will be using a mobile game "Egg, Inc." as the target for
17demonstration. It's a simple time killer app that got me through boring long waits when I was still at school.
18
1e75f355
JM
19Egg, Inc. is a basic incremental "idler" game where your goal is to take over the world food supply with ever-increasing supply of eggs,
20if you have ever played Cookie Clicker, you know the premise of something like that. You have to unlock denser and denser eggs - the game
21is also designed around the fact that you can do certain online-tied activites such as Contracts to unlock more Soul Eggs (prestige boost) and
22"Eggs of Prophecy" which increase potency of your Soul Eggs.
23
24It's rather simple game with a very minimal API, making it perfect for learning. You may not like the game, but that's beside the point.
25The simplicity of our target matters here.
2a5bb5a5
JM
26
27## The existing works
1e75f355
JM
28In some cases, you will find previous works on the target you pick. In my case, some clever people have created
29[scripts to extract .proto file out of the app.](https://github.com/DavidArthurCole/EggIncProtoExtractor)
2a5bb5a5
JM
30I advise you to check it out if you wish to get a better understanding of how you would go about retrieving the
31API spec .proto file for your target.
32
33Further there are a few dedicated individuals in the game's community who have created numerous tools and historical databases.
34
35For this blog purposes, we will assume the game server is shut down (as in we cannot query from the live API) and our goal is
36to make a semi-functional selfhosted gameserver for our own needs, assuming we are the only one on said server.
37
1e75f355
JM
38## How to source builds of a game
39There are two methods of sourcing the apk file here - one method is if you already have the app installed, install something like ZArchiver
40and extract it from /data/app/ - identifying the app by its icon. From there you will find `base.apk` which is enough for most apps.
41
42Alternatively, if the app is still available on Google Play, you can use an app like Aurora Store to go to the store detail page, select
43"Manual Download" and enter a known Build ID.
44
2a5bb5a5
JM
45## Getting Started
46Thanks to the previously mentioned script, it's easy to get started - find the APK, extract protobuf spec file, convert it with
47protoc and we're done there. One small problem - due to cheaters, latest version of the game includes "AuthenticatedMessage" structure,
48which contains a salted sha256sum of the payload message.
49
50At this point, after a bit of internal dilemma, I decided to not further the problem while service is still live for people playing and did the
51more morally sound decision of picking a version prior to these integrity checks. We can crack that another day as all the needed information
52is retained in the app itself.
53
1e75f355 54Going forward with this, we are targetting game version 1.12.13 (Build ID 111121 - use that in Aurora Store).
2a5bb5a5
JM
55
56With all that out of the way, lets get into actual commands used here:
57```
58git clone https://github.com/DavidArthurCole/EggIncProtoExtractor.git
59cd EggIncProtoExtractor
60./apkextract.sh com.auxbrain.egginc_1.12.13.apk
61# We should have a new folder "protos" now with resulting files
62cd protos
63# There should be a file called ei.proto - that's our protobuf spec file
64# At this point, we can use the protoc utility which can convert the specfile
65# to interfaces in C++, C#, Java, Kotlin, Objective-C, PHP, Python and Ruby with
66# additional plugin support for Dart and Go.
67# To make this easier to understand, we will use Python in this demonstration
68protoc -I=. --python_out=. ./ei.proto
69# Success! We now have a "ei_pb2.py" file which can be directly imported to Python programs
70```
71
72With the protobuf interface in Python created, we can now proceed with creating the API emulator - but there's a slight problem.
73What URL? What endpoints? How do we find this out? Simple answer, disassembling the game. Get your RE tool of choice, I will be
74using [Ghidra](https://ghidra-sre.org/) myself.
75
76(Note: You can also just try to find this out using tools such as WireShark)
77
78The game contains a linked-library written in C++, which you can find inside the .apk `lib` folder, named as `libegginc.so`.
79This is perfect for our use-case, Ghidra is going to slice through this like butter. Import the file to your RE tool of choice
80and let it perform some analysis on it, have a cup of tea or coffee as this is going to take a hot minute.
81
82Once that's done, we are going to start by looking at the defined strings - try our luck there. Search for any debug prints left
83behind or maybe some clues. I started by searching for `http`, which lead me to following string `"HTTP REQ: %d"`, seems promising.
84When I jumped to it, I saw an exactly adjacent string to it which could give more clues:
85```
86 s_www.auxbrain.com_00c02b60 XREF[0,1]: FUN_00518ab8:00518b38(R)
87 00c02b5e 47 3f 77 ds "G?www.auxbrain.com"
88 77 77 2e
89 61 75 78
90 s_HTTP_REQ:_%d_00c02b71 XREF[1]: makeRequestInternal:0067bbd4(*)
91 00c02b71 48 54 54 ds "HTTP REQ: %d"
92 50 20 52
93 45 51 3a
94```
95Interesting, `www.auxbrain.com`. If we jump to its XREF, we get a garbled function, but what it seems to be doing is setting up
96certain global values.
97
be8f6a91
JM
98## The smoke-test
99
2a5bb5a5
JM
100So we have a potential API endpoint, let's put it to the test. We're not going to recompile anything yet or do any byte-patching,
101let's try a quick smoke-test. Ensure your phone is rooted and you have a variant of Xposed Framework installed (I used LSPosed).
1e75f355 102We will need to unarm the SSL pinning present in most apps, including this one, I used [io.github.tehcneko.sslunpinning](https://github.com/Xposed-Modules-Repo/io.github.tehcneko.sslunpinning) module.
9656149b 103(NOTE: Users without root might want to skip to end of article where I showcase unpinning the app manually)
2a5bb5a5
JM
104
105Next, install [AdAway app from F-Droid](https://f-droid.org/packages/org.adaway/) so we can setup a redirection on any network we are on.
106Inside AdAway, add a redirection rule for the address we just found and point it to an IP address in your LAN that will run the API server.
107
108Generate a self-signed certificate authority and a certificate signed by it and run a webserver with both HTTP and HTTPS on the API server machine.
be8f6a91
JM
109```
110# Create an ext file containing the Subject Alternative Name (SAN)
9656149b
JM
111# DNS.1 should correspond to the API endpoint of the app.
112# NOTE! If you are changing the API endpoint to a public domain, you can just use a public cert, no need for any of this.
be8f6a91
JM
113cat > auxbrain.ext << EOF
114authorityKeyIdentifier=keyid,issuer
115basicConstraints=CA:FALSE
116keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
117subjectAltName = @alt_names
118
119[alt_names]
120DNS.1 = www.auxbrain.com
121EOF
122
123# Create your own Certificate Authority
124openssl genrsa -des3 -out myCA.key 2048
125openssl req -x509 -new -nodes -key myCA.key -sha256 -days 1825 -out myCA.pem
126# Create a CSR and lets have the new CA sign it
3bcb1df5 127openssl genrsa -out auxbrain.key 2048
be8f6a91 128openssl req -new -key auxbrain.key -out auxbrain.csr -nodes
f055cad6
JM
129openssl x509 -req -in auxbrain.csr -CA myCA.pem -CAkey myCA.key -CAcreateserial -out auxbrain.crt -days 825 -sha256 -extfile auxbrain.ext
130cat auxbrain.crt myCA.pem > auxbrain.pem
596663d8
JM
131# You now have:
132# myCA.pem - the public certificate of your root CA
133# auxbrain.key - the private key for your webserver
134# auxbrain.pem - the public cert for your webserver.
be8f6a91
JM
135```
136
137Use the generated `auxbrain.pem` and `auxbrain.key` files for your webserver SSL/TLS configuration. For nginx, append following values to your server directive:
138```
139listen 443 ssl;
140ssl_certificate /path/to/auxbrain.pem;
141ssl_certificate_key /path/to/auxbrain.key;
142ssl_session_cache shared:SSL:1m;
143ssl_session_timeout 5m;
144ssl_ciphers HIGH:!aNULL:!MD5;
145ssl_prefer_server_ciphers on;
146```
147
148Import the self-signed CA (myCA.pem) to your phone's truststore (Check under your phone's Security/Encryption settings). Once all of that is done, run the app for first time.
2a5bb5a5
JM
149
150```
151192.168.1.212 - - [...] "POST /ei/first_contact HTTP/1.1" 404 0 "-"
152```
153
154Bingo. We have contact and we have an API endpoint. Searching for "ei/" in the strings reveals a extensive list of API endpoints, we now have something
155to go off from. We have everything we need to start creating the server.
156
157## Implementing the Server - Getting first contact
158Next up, we create a new project - as we generated the protobuf definitions for Python, we will proceed accordingly.
159If you are following along, get respective packages for your operating system to create python venvs.
160As the protobufs are being sent over HTTP, we will be serving our application over flask which is being reverse proxied by nginx.
161
162```
163# Lets stage the environment
164mkdir apiserver
165cd apiserver
166python -m venv .venv
167source .venv/bin/activate
168touch app.py
169cp ~/EggIncProtoExtractor/protos/ei.proto .
170
171# Get some dependeices
172pip install protobuf
173pip install flask
174```
175
176We now have the project set up for reading protobuf definitions and a framework to listen for HTTP and routes sent to it.
177Let's create an actual listener application, open app.py with your favourite IDE or text editor.
178
179```
180import ei_pb2 as EIProto
181from flask import Flask
182from flask import request
183
184@app.route("/ei/<path:subpath>", methods=["POST"])
185def ei_routes(subpath):
186 print("HTTP POST /ei/" + subpath)
187 print(request.headers)
188 return ""
189```
190
191This should get the ball rolling, we will see whatever call comes in and we can see what the payload of each request contains.
192At this point you should setup the reverse proxy, override your nginx / directive with:
193```
194location / {
195 proxy_pass http://127.0.0.1:5000;
196}
197```
198
199Reload your nginx and start the flask application you just created with `flask run`.
200
201Run the app again and have it phone home and see what it contains.
202```
203HTTP POST /ei/first_contact
204Host: 127.0.0.1:5000
205Connection: close
206Content-Length: 37
207Content-Type: application/x-www-form-urlencoded
208User-Agent: Dalvik/2.1.0 (Linux; U; Android 13; M2012K11AG Build/TQ3A.230901.001)
209Accept-Encoding: gzip
210```
211
212We can see there's a form payload attached to this request, let's modify our app route a bit:
213```
214@app.route("/ei/<path:subpath>", methods=["POST"])
215def ei_routes(subpath):
216 print("HTTP POST /ei/" + subpath)
217 print(request.form)
218 return ""
219```
220
221Now if we run the modified flask application again, we see following output on the first_contact endpoint.
222```
223HTTP POST /ei/first_contact
224ImmutableMultiDict([('data', 'ChAzNTVlNDZlOTA4OWQxZTRjEAAYAg==')])
225```
226
227We have a base64-encoded protobuf binary data - which isn't terribly useful for reading plain-text, since protobuf *is* a binary
228format, so we will need to figure out what protobuf message this payload belongs to.
229
230Remember that ei.proto file alongside the ei_pb2.py we got earlier? Lets go back there and inspect it a bit.
231We know we just contacted something called "first_contact", maybe there is something in that file that could help us?
232
233```
234message EggIncFirstContactRequest {
235 optional string user_id = 1;
236 optional uint32 client_version = 2;
237 optional Platform platform = 3;
238}
239
240message EggIncFirstContactResponse {
241 optional Backup backup = 1;
242}
243```
244
245Seems like the application is using message names in almost similar fashion to API endpoint names themselves. This will prove
246to be useful knowledge. We now know what the payload should be, lets put this to the test.
247
248Edit your app routine again
249```
250# add "import base64" to top of the file
251@app.route("/ei/<path:subpath>", methods=["POST"])
252def ei_routes(subpath):
253 print("HTTP POST /ei/" + subpath)
254 if subpath == "first_contact":
255 # Create the protobuf object so we can load data from the b64 payload
256 FirstContact = EIProto.EggIncFirstContactRequest()
257 FirstContact.ParseFromString(base64.b64decode(form["data"]))
258 print(FirstContact)
259 else:
260 print(request.form)
261 return ""
262```
263
264We should now be able to see deserialized output when we run the flask application and the mobile app, let's try it out:
265
266```
267HTTP POST /ei/first_contact
268user_id: "355e46e9089d1e4c"
269client_version: 0
270platform: DROID
271```
272
273Nice! We now know how to identify which protobuf object corresponds to which API endpoint. We can now make an educated guess
274on what would come next.
275
276Seeing how we got `EggIncFirstContactRequest` and saw an adjacent `EggIncFirstContactResponse` message in the proto file, we
277can safely assume that this is what the game is expecting from us in return.
278
279Lets modify the server a bit to account for that.
280
281```
282@app.route("/ei/<path:subpath>", methods=["POST"])
283def ei_routes(subpath):
284 print("HTTP POST /ei/" + subpath)
285 if subpath == "first_contact":
286 # Create the protobuf object so we can load data from the b64 payload
287 FirstContact = EIProto.EggIncFirstContactRequest()
288 FirstContact.ParseFromString(base64.b64decode(form["data"]))
289 print("We got a first contact hello from user " + FirstContact.user_id)
290 # Lets respond with a FirstContactResponse
291 FirstContactResp = EIProto.EggIncFirstContactResponse()
292 # This takes only one optional argument - a Backup object - but we have no account
293 # system yet, so we will opt out of sending that for now.
294 # ---
295 # We send the payload back as a base64 string - the same way we retrieved it.
296 return base64.b64encode(FirstContactResp.SerializeToString())
297 else:
298 print(request.form)
299 return ""
300```
301
302Now when we run the app again, we notice that we no longer get spammed this endpoint, but instead in its place we see a few new friends
303
304## Implementing the Server - New Friends
305Say hello to `/ei/save_backup` and `/ei/get_periodicals`. We can infer from the name, that save_backup would involve a Backup message
306and get_periodicals would involve a GetPeriodicalsRequest, both of which are defined fully in the proto spec file.
307
308Both of these are clogging up the flask application log periodically, we should check out what they are so we can have a sane log again.
309
310```
311@app.route("/ei/<path:subpath>", methods=["POST"])
312def ei_routes(subpath):
313 print("HTTP POST /ei/" + subpath)
314 if subpath == "first_contact":
315 # Create the protobuf object so we can load data from the b64 payload
316 FirstContact = EIProto.EggIncFirstContactRequest()
317 FirstContact.ParseFromString(base64.b64decode(form["data"]))
318 print("We got a first contact hello from user " + FirstContact.user_id)
319 # Lets respond with a FirstContactResponse
320 FirstContactResp = EIProto.EggIncFirstContactResponse()
321 # This takes only one optional argument - a Backup object - but we have no account
322 # system yet, so we will opt out of sending that for now.
323 # ---
324 # We send the payload back as a base64 string - the same way we retrieved it.
325 return base64.b64encode(FirstContactResp.SerializeToString())
326 elif subpath == "save_backup":
327 # NOTE: This took me way longer to realize than it should have, but the base64
328 # payload you receive from client is broken due to some Android bug, where it
329 # substitutes "+" symbols with a " " whitespace.
330 # I don't want you to waste half hour to hours figuring out why you're getting
331 # corrupted data, so you're welcome.
332 Backup = EIProto.Backup()
333 Backup.ParseFromString(base64.b64decode(form["data"].replace(" ", "+"))
334 print(Backup)
335 elif subpath == "get_periodicals":
336 Periodicals = EIProto.GetPeriodicalsRequest()
337 Periodicals.ParseFromString(base64.b64decode(form["data"])
338 print(Periodicals)
339 else:
340 print(request.form)
341 return ""
342```
343
344We should now see what these payloads actually contain when deserialized (for your reading experience, I advise you to rather
345try this out yourself - the Backup structure is VERY large).
346
347Upon these payloads reaching the server, we see that a very much populated Backup message makes its way through and a relatively
348thin payload of PeriodicalsRequest comes through, which isn't very useful by itself.
349
350Now, this is why the game developer ended up creating forms of anticheat in future versions of this app - the Backup message contains
351your entire game state, which is often sent as a way to save your progress to cloud, but there is no actual sanity checking in place
352to ensure you're not progressing way too fast. Personally, I am of the mind that anticheat should be done on the server-side, not
353on the client-side, but I digress. We can use this to prove a very obvious vulnerability when using trust-client-always architecture.
354
355The game has an In App Purchase for "Pro Permit", which allows you to build more Silos, which in turn allow you to get offline
356rewards for a longer period of time. If we look at protobuf definition file, you can see under Backup.game, a field called "permit_level",
357which by default is zero. Lets try to change that and present a modified Backup the next time user opens the game.
358
359
360```
361cache = {}
362
363@app.route("/ei/<path:subpath>", methods=["POST"])
364def ei_routes(subpath):
365 print("HTTP POST /ei/" + subpath)
366 if subpath == "first_contact":
367 # Create the protobuf object so we can load data from the b64 payload
368 FirstContact = EIProto.EggIncFirstContactRequest()
369 FirstContact.ParseFromString(base64.b64decode(form["data"]))
370 print("We got a first contact hello from user " + FirstContact.user_id)
371 # Lets respond with a FirstContactResponse
372 FirstContactResp = EIProto.EggIncFirstContactResponse()
373 if FirstContact.user_id in cache:
374 FirstContactResp.backup.CopyFrom(cache[FirstContact.user_id])
375 del cache[FirstContact.user_id]
376 return base64.b64encode(FirstContactResp.SerializeToString())
377 elif subpath == "save_backup":
378 # NOTE: This took me way longer to realize than it should have, but the base64
379 # payload you receive from client is broken due to some Android bug, where it
380 # substitutes "+" symbols with a " " whitespace.
381 # I don't want you to waste half hour to hours figuring out why you're getting
382 # corrupted data, so you're welcome.
383 Backup = EIProto.Backup()
384 Backup.ParseFromString(base64.b64decode(form["data"].replace(" ", "+"))
385 if Backup.game.permit_level == 0:
386 print("Saved a modified Backup for next game load")
387 # Modify the permit level, force offer the backup
388 Backup.game.permit_level = 1
389 Backup.force_offer_backup = True
390 Backup.force_backup = True
391 cache[Backup.user_id] = Backup
392 elif subpath == "get_periodicals":
393 Periodicals = EIProto.GetPeriodicalsRequest()
394 Periodicals.ParseFromString(base64.b64decode(form["data"])
395 print(Periodicals)
396 else:
397 print(request.form)
398 return ""
399```
400
401Lets load up the game. Nothing interesting seems to be happening yet - lets wait until we see the "Saved a modified Backup for next game load" message
402show up in the server console. Once this shows up, restart the game - you are presented with a popup that you are offered to load a
403Backup from server. Let's accept that.
404
405Now click on your silos, you have the Pro Permit for free.
406
407Now, it goes without saying, I do not condone piracy - the future versions of this game are very much guarded against this, rightfully so.
408If you attempt this in actual game servers, this is considered fraud and IS detectable by the developer (every IAP has a receipt, logically!).
409
410This version of the game is defunct as the protocol has changed quite a bit in the years since this version and additional anticheat
411measures have been added since. You cannot transfer this status (or even purchase what you just did) from this game version to the next.
412
413### Onto the PeriodicalsRequest
414This one is a bit more fun to delve into blindly - the proto spec wont help you much here. We will need to use our trusty RE tools again and delve into
415the game disassembly again.
416
417By public knowledge, we know there are server events for "Epic Research Sale", "Research Sale", "Drone Bonus" and "Prestige Boost". We can use this information to try
418and look at any potential leads in strings. Drone seems like a good canditate, lets look into that.
419```
420drone_fans2
421drone_crash
422drone_enemy
423drone_hunter
424r_icon_drone_rewards
425b_icon_drone_boost
426drone_touch
427ei_drone_lights_green
428ei_drone_lights_red
429ei_drone_package
430ei_drone_propeller
431drone-boost
432GENEROUS DRONES
433```
434This looks promising, right off the bat, first strings I'd check here are `r_icon_drone_rewards`, `b_icon_drone_boost`, `drone-boost` and `GENEROUS DRONES`.
435
1e75f355 436I inspected all 4 of them, and when I got to the final 2, I found the enum string translations used for event IDs - here they are extracted for game version 1.12.13
2a5bb5a5
JM
437```
438piggy-boost (Rate piggy fills is increased.)
439piggy-cap-boost (UNLIMITED PIGGY;Gains are retained when event ends.)
440prestige-boost (PRESTIGE BOOST;Collect more soul eggs on prestige, you must prestige to take advantage of this event.)
441earnings-boost (CASH BOOST;Regular earnings are increased.)
442gift-boost (GENEROUS GIFTS;Boost applies to random gifts and video gifts.)
443drone-boost (GENEROUS DRONES;Drones will produce larger rewards.)
444epic-research-sale (EPIC RESEARCH SALE;Only applies to Epic Research.)
445vehicle-sale (VEHICLE SALE;Applies to all vehicles.)
446boost-sale (BOOST SALE;Applies to the gold price of boosts.)
447boost-duration (BOOST TIME+;Boosts last longer, you must start a boost during the event.)
448```
449I recall there being a few more boosts, but this is useful for getting started with compositing PeriodicalsResponse with an active running event.
450
451### Putting together the response
452We have the enum, we have the names, descriptions, lets try to create a sample server event when the client enqueries about current server periodical events.
453```
454 elif subpath == "get_periodicals":
455 # We don't actually need the information client sends us,
456 # we aren't verifying any stats about client in our server.
457 CurrentPeriodicals = EIProto.PeriodicalsResponse()
458 # In order to add items to a repeatable field in protobuf structure,
459 # we need to call .add() method on it
460 event = CurrentPeriodicals.events.events.add()
461 # Refer to ei.proto - we are filling fields for EggIncEvent structure here.
462 event.type = "drone-boost"
463 event.multiplier = 5.00
464 event.subtitle = "Drones will produce larger rewards."
465 event.identifier = "GENEROUS DRONES"
466 event.seconds_remaining = 300.0
467 # Lets make it respond with a 5 minute event (this will re-arm itself when client calls
468 # for get_periodicals again every 6 minutes)
469 return base64.b64encode(CurrentPeriodicals.SerializeToString())
470```
471
472Launch the server and observe as the client periodically calls this endpoint again, it will now receive a 5 minute 5x Drone Rewards boost on the game.
473
474## Created the Server - What now?
475We have now created a very basic server, which appropriately responds to a first contact, misuses the game backup feature to prove a point about weaknesses of
476trusting client in server. We also created a very basic server event, which always rearms itself to never expire.
477
478What do we do next?
479
480At this point, we can start dog-fooding the project. Lets start with whatever ball game throws at us as we progress.
481
482### Contracts
483As we progress the game and start performing prestiges, we unlock a feature called "Contracts" - but disaster strikes as we don't have any contracts we could
1e75f355 484accept. So far we still see our good friends `/ei/get_periodicals` and `/ei/save_backup` hammering the server at regular intervals.
2a5bb5a5
JM
485
486When we created the periodicals response payload, you might have noticed in the protobuf message an optional field called `ContractsResponse contracts`. Lets see
487what this ContractsResponse message contains.
488
489```
490message ContractsResponse {
491 repeated Contract contracts = 1;
492 optional string warning_message = 4;
493 optional double server_time = 2;
494 optional uint32 max_eop = 3 [default = 1000];
495}
496```
497
498Notice there being an array of Contract messages right off the bat - lets find its message structure next:
499
500```
501message Contract {
502 optional string identifier = 1;
503 optional string name = 9;
504 optional string description = 10;
505 optional Egg egg = 2;
506
507 repeated Goal goals = 3;
508 message Goal {
509 optional GoalType type = 1;
510 optional double target_amount = 2;
511 optional RewardType reward_type = 3;
512 optional string reward_sub_type = 4;
513 optional double reward_amount = 5;
514 optional double target_soul_eggs = 6;
515 }
516
517 repeated GoalSet goal_sets = 16;
518 message GoalSet {
519 repeated Goal goals = 1;
520 }
521
522 optional bool coop_allowed = 4;
523 optional uint32 max_coop_size = 5;
524 optional uint32 max_boosts = 12;
525 optional double minutes_per_token = 15 [default = 60];
526 optional double expiration_time = 6;
527 optional double length_seconds = 7;
528 optional double max_soul_eggs = 13;
529 optional uint32 min_client_version = 14;
530 optional bool debug = 11;
531}
532```
533
534We will need to do a bit of reading. Fortunately, the game has a community wiki, lets look into how contracts should work. I took an
535[older revision of Contracts wiki page from 2021](https://egg-inc.fandom.com/wiki/Contracts?oldid=13015) and did some slight research.
536
537From what I gather, at one point, there was only one set of contract rewards, shared between everyone - then they created a system where
538beginners will get easier contract goals while more advanced players get harder contract goals.
539
540We can put two-and-two together here and infer that `repeated Goal goals` is the legacy contract system - where everyone was on equal footing
541and `repeated GoalSet goal_sets` is the *new* goal system that is split into Standard and Elite.
542
1e75f355 543We also learn that in future game versions, they completely reworked how contracts work *yet* again into a grading "bracket" system. Fortunately,
2a5bb5a5
JM
544we do not have to worry about that in our current target revision.
545
546Now to get the ball rolling, there is conveniently a starting point set ahead for us already. The developer of game intended to ease new players into
547contracts by creating a simple & easy contract called [Your First Contract](https://egg-inc.fandom.com/wiki/Contracts/Your_First_Contract?oldid=13547).
548
549This page tells us all the information we need to compose our first Contract, so lets try to make one.
550
551```
552 elif subpath == "get_periodicals":
553 # We don't actually need the information client sends us,
554 # we aren't verifying any stats about client in our server.
555 CurrentPeriodicals = EIProto.PeriodicalsResponse()
556 # [...]
557 Contract = CurrentPeriodicals.contracts.contracts.add()
558 Contract.identifier = "first-contract"
559 Contract.name = "Your First Contract"
560 Contract.description = "We heard you are open to contract work! Help fill this order from the local pharmacy!"
561 Contract.egg = EIProto.Egg.MEDICAL
562 Contract.coop_allowed = False
563 Contract.minutes_per_token = 5
564 # Lets set expiry time to always be 3 days into future
565 Contract.expiration_time = time.time() + (3600.0 * 72.0)
566 Contract.length_seconds = 3600.0 * 4.0
567 # The wiki mentions that you cannot get this contract after you reach 5000 Soul Eggs
568 Contract.max_soul_eggs = 5000.0
569 # We should have the basic metadata set now, lets create the goalsets.
570 FirstSet = Contract.goal_sets.add()
571 Goal = FirstSet.goals.add()
572 # There is only one type of goal in this verison
573 Goal.type = EIProto.GoalType.EGGS_LAID
574 Goal.target_amount = 100000.0
575 Goal.reward_type = EIProto.RewardType.GOLD
576 Goal.reward_amount = 192
577 Goal = FirstSet.goals.add()
578 Goal.type = EIProto.GoalType.EGGS_LAID
579 Goal.target_amount = 500000000.0
580 Goal.reward_type = EIProto.RewardType.PIGGY_FILL
581 Goal.reward_amount = 10000
582 # Lets now add the Elite table, we can pretty much copy-paste the above here.
583 SecondSet = Contract.goal_sets.add()
584 Goal = SecondSet.goals.add()
585 Goal.type = EIProto.GoalType.EGGS_LAID
586 Goal.target_amount = 100000.0
587 Goal.reward_type = EIProto.RewardType.GOLD
588 Goal.reward_amount = 500
589 Goal = SecondSet.goals.add()
590 Goal.type = EIProto.GoalType.EGGS_LAID
591 Goal.target_amount = 500000000.0
592 Goal.reward_type = EIProto.RewardType.PIGGY_FILL
593 Goal.reward_amount = 10000
594 return base64.b64encode(CurrentPeriodicals.SerializeToString())
595```
596
1e75f355 597Lets try that out in-game now - after waiting for a minute, we see our contract pop up, but I immediately noticed one thing amiss.
2a5bb5a5
JM
598The contract goals are swapped! I am getting Elite contract rewards for a Standard contract.
599
600This piece of information now tells us that the first entry in GoalSets refers to Elite rewards and the second entry in GoalSets to Standard rewards.
601After swapping the sets around, we now see a contract with the corrected rewards.
602
603I playtested it a bit and the contract worked as expected.
604
605Now, the above code could be a lot neater. For your homework, if you're not skipping to the public source release in the end, you should try to create
606a contract database and try scheduling them like the game originally did - a "Leggacy" contract every Friday and regular contracts showing up every 1-2 weeks
607for roughly 2 weeks.
608
1e75f355
JM
609## Rootless SSL Unpinning + Endpoint URL patching
610Let's make the app not require a VPN or root privileges - let's make user CAs work and the endpoint URL something we control on the public net.
611Start off by pulling the following repository
612```
613git clone https://github.com/ilya-kozyr/android-ssl-pinning-bypass.git
614python3 -m venv .venv
615source .venv/bin/activate
616pip install -r requirements.txt
617cp /path/to/your/apk .
618python3 apk-rebuild.py egginc.apk --pause
619```
620
9656149b
JM
621**NOTE!** IF you do not intend to patch the API endpoint and just want to proceed with AdAway redirecting traffic, you can stop here and press ENTER!
622Proceed only if you own a domain in your control (that is equal or less in length to www.auxbrain.com) and want to use the app without a VPN/redirection.
623
1e75f355
JM
624Open a new terminal window, the script will wait for us to perform modifications, enter the created folder `egginc.apk-decompiled` and `lib`.
625
626We have two folders here now, `arm64-v8a` and `armeabi-v7a`, just as we saw when we pulled the .so file out of the apk earlier. Let's tackle
627the 64-bit build first.
628
629For arm64 build it was really simple to perform bytepatch on the said endpoint. We already know it's supposed to look as `G?www.auxbrain.com` - let's probe the .so library a bit.
630```
631$> hexdump -C libegginc.so | grep "ww.auxbrain.co" -A2 -B2
63200b02b40 cd cc 4c 3f 00 00 00 00 00 00 00 00 00 00 80 3f |..L?...........?|
63300b02b50 00 00 00 00 00 00 00 00 00 00 00 00 14 ae 47 3f |..............G?|
63400b02b60 77 77 77 2e 61 75 78 62 72 61 69 6e 2e 63 6f 6d |www.auxbrain.com|
63500b02b70 00 48 54 54 50 20 52 45 51 3a 20 25 64 00 64 61 |.HTTP REQ: %d.da|
63600b02b80 74 61 3d 00 65 69 2f 66 69 72 73 74 5f 63 6f 6e |ta=.ei/first_con
637```
638
639We seem to have nothing blocking our way, let's create hex representations of `G?www.auxbrain.com` and a target domain of equal length, for example `G?eggs.based.quest`.
640
641(Note: You can choose a shorter name as well, if you null-terminate the extra bytes as padding)
642```
643$> echo "G?www.auxbrain.com" | hexdump -ve '1/1 "%.2X"'
644473F7777772E617578627261696E2E636F6D0A
645$> echo "G?eggs.based.quest" | hexdump -ve '1/1 "%.2X"'
646473F656767732E62617365642E71756573740A
647```
648
649Remove the trailing `0A` from end of both hex strings and now proceed as follows:
650```
651# Place the source in first bracket of sed and the new URL at second bracket.
652hexdump -ve '1/1 "%.2X"' libegginc.so | sed "s/473F7777772E617578627261696E2E636F6D/473F656767732E62617365642E7175657374/g" | xxd -r -p > patched.so
653```
654
655Huzzah! We now have a patched linked-library for the arm64 build. Let's also patch the 32-bit version.
656```
657$> hexdump -C libegginc.so | grep "ww.auxbrain.co" -A2 -B2
6580087b770 69 67 68 5f 74 6f 6f 5f 6d 61 6e 79 5f 70 78 00 |igh_too_many_px.|
6590087b780 74 61 62 6c 65 74 5f 68 64 70 69 00 00 00 00 00 |tablet_hdpi.....|
6600087b790 77 77 77 2e 61 75 78 62 72 61 69 6e 2e 63 6f 6d |www.auxbrain.com|
6610087b7a0 00 00 00 00 00 00 00 00 65 69 2f 66 69 72 73 74 |........ei/first|
6620087b7b0 5f 63 6f 6e 74 61 63 74 00 00 00 00 00 00 00 00 |_contact........|
663```
664This one lacks the `G?` prefix on API endpoint, but we still have null terminators we can rely on. Let's replace the `473F` from our previous strings with `0000`.
665```
666# Place the source in first bracket of sed and the new URL at second bracket.
667hexdump -ve '1/1 "%.2X"' libegginc.so | sed "s/00007777772E617578627261696E2E636F6D/0000656767732E62617365642E7175657374/g" | xxd -r -p > patched.so
668```
669
670Replace both of the libegginc.so files with the patched.so files. Move back to main terminal window and press ENTER.
671
672We now have a patched and debug signed apk for the game that isn't SSL pinned and contains a custom API endpoint we control without a VPN.
673
2a5bb5a5
JM
674## Conclusion so far
675We have created a (rather ugly looking) server emulator for the game. It functions, but it needs a lot of work still before we can call it ready.
676If you have followed this far, give yourself pat on the back - if you actually tried to run this code, give yourself an extra pat on the back.
677
678Before I give you the public source to the project, you might want to try your hand at creating a few more things.
679- "Cloud" save, present a Backup to any new device that just started playing.
680- Contracts Database and scheduler
681- Server Event scheduler
682
683I apologize if my method of documenting this has been messy, but that's also part of the chaos of reverse engineering, you are constantly learning new things
684about the project you are currently doing - refactoring becomes an essential part once you have documented the protocol to a comfortable degree.
685
686I won't give any promises for a part 2 any time soon, but I will be trying to make this feature complete, so without further ado, here are the git repository links:
687[github.com](https://github.com/cernodile/reEgg), [git.based.quest](https://git.based.quest/?p=reEgg.git;a=tree;h=refs/heads/master;hb=refs/heads/master).
688
689Thank you for reading and making it all the way to the end,
690- Cernodile