Commit | Line | Data |
---|---|---|
2a5bb5a5 JM |
1 | --- |
2 | title: "Reverse Engineering a Mobile App Protobuf API" | |
3 | date: 2024-05-11T12:00:00+03:00 | |
4 | description: "In recent times more than ever, live service and overuse of APIs soon-to-be-stale is only increasing. This causes a lot of once-written software to become either unusable or handicapped in many regards. One way to fight this is to learn reverse engineering for sake of digital preservation. In this blog, I take you through a journey of reverse engineering a simple mobile game's protobuf API." | |
5 | tags: ['tutorial', 'reverse-engineering', 'opensource'] | |
6 | type: blog | |
7 | draft: false | |
8 | --- | |
9 | ||
10 | # Why | |
11 | Why not? Digital preservation is important, even if you don't care for a specific program. | |
12 | This is also a good way to get started with protocol reverse engineering due to way protobuf | |
13 | is often left behind in source format within client applications. | |
14 | ||
15 | ## The target | |
16 | In this series of blogposts, I will be using a mobile game "Egg, Inc." as the target for | |
17 | demonstration. It's a simple time killer app that got me through boring long waits when I was still at school. | |
18 | ||
1e75f355 JM |
19 | Egg, Inc. is a basic incremental "idler" game where your goal is to take over the world food supply with ever-increasing supply of eggs, |
20 | if you have ever played Cookie Clicker, you know the premise of something like that. You have to unlock denser and denser eggs - the game | |
21 | is also designed around the fact that you can do certain online-tied activites such as Contracts to unlock more Soul Eggs (prestige boost) and | |
22 | "Eggs of Prophecy" which increase potency of your Soul Eggs. | |
23 | ||
24 | It's rather simple game with a very minimal API, making it perfect for learning. You may not like the game, but that's beside the point. | |
25 | The simplicity of our target matters here. | |
2a5bb5a5 JM |
26 | |
27 | ## The existing works | |
1e75f355 JM |
28 | In some cases, you will find previous works on the target you pick. In my case, some clever people have created |
29 | [scripts to extract .proto file out of the app.](https://github.com/DavidArthurCole/EggIncProtoExtractor) | |
2a5bb5a5 JM |
30 | I advise you to check it out if you wish to get a better understanding of how you would go about retrieving the |
31 | API spec .proto file for your target. | |
32 | ||
33 | Further there are a few dedicated individuals in the game's community who have created numerous tools and historical databases. | |
34 | ||
35 | For this blog purposes, we will assume the game server is shut down (as in we cannot query from the live API) and our goal is | |
36 | to make a semi-functional selfhosted gameserver for our own needs, assuming we are the only one on said server. | |
37 | ||
1e75f355 JM |
38 | ## How to source builds of a game |
39 | There are two methods of sourcing the apk file here - one method is if you already have the app installed, install something like ZArchiver | |
40 | and extract it from /data/app/ - identifying the app by its icon. From there you will find `base.apk` which is enough for most apps. | |
41 | ||
42 | Alternatively, if the app is still available on Google Play, you can use an app like Aurora Store to go to the store detail page, select | |
43 | "Manual Download" and enter a known Build ID. | |
44 | ||
2a5bb5a5 JM |
45 | ## Getting Started |
46 | Thanks to the previously mentioned script, it's easy to get started - find the APK, extract protobuf spec file, convert it with | |
47 | protoc and we're done there. One small problem - due to cheaters, latest version of the game includes "AuthenticatedMessage" structure, | |
48 | which contains a salted sha256sum of the payload message. | |
49 | ||
50 | At this point, after a bit of internal dilemma, I decided to not further the problem while service is still live for people playing and did the | |
51 | more morally sound decision of picking a version prior to these integrity checks. We can crack that another day as all the needed information | |
52 | is retained in the app itself. | |
53 | ||
1e75f355 | 54 | Going forward with this, we are targetting game version 1.12.13 (Build ID 111121 - use that in Aurora Store). |
2a5bb5a5 JM |
55 | |
56 | With all that out of the way, lets get into actual commands used here: | |
57 | ``` | |
58 | git clone https://github.com/DavidArthurCole/EggIncProtoExtractor.git | |
59 | cd EggIncProtoExtractor | |
60 | ./apkextract.sh com.auxbrain.egginc_1.12.13.apk | |
61 | # We should have a new folder "protos" now with resulting files | |
62 | cd protos | |
63 | # There should be a file called ei.proto - that's our protobuf spec file | |
64 | # At this point, we can use the protoc utility which can convert the specfile | |
65 | # to interfaces in C++, C#, Java, Kotlin, Objective-C, PHP, Python and Ruby with | |
66 | # additional plugin support for Dart and Go. | |
67 | # To make this easier to understand, we will use Python in this demonstration | |
68 | protoc -I=. --python_out=. ./ei.proto | |
69 | # Success! We now have a "ei_pb2.py" file which can be directly imported to Python programs | |
70 | ``` | |
71 | ||
72 | With the protobuf interface in Python created, we can now proceed with creating the API emulator - but there's a slight problem. | |
73 | What URL? What endpoints? How do we find this out? Simple answer, disassembling the game. Get your RE tool of choice, I will be | |
74 | using [Ghidra](https://ghidra-sre.org/) myself. | |
75 | ||
76 | (Note: You can also just try to find this out using tools such as WireShark) | |
77 | ||
78 | The game contains a linked-library written in C++, which you can find inside the .apk `lib` folder, named as `libegginc.so`. | |
79 | This is perfect for our use-case, Ghidra is going to slice through this like butter. Import the file to your RE tool of choice | |
80 | and let it perform some analysis on it, have a cup of tea or coffee as this is going to take a hot minute. | |
81 | ||
82 | Once that's done, we are going to start by looking at the defined strings - try our luck there. Search for any debug prints left | |
83 | behind or maybe some clues. I started by searching for `http`, which lead me to following string `"HTTP REQ: %d"`, seems promising. | |
84 | When I jumped to it, I saw an exactly adjacent string to it which could give more clues: | |
85 | ``` | |
86 | s_www.auxbrain.com_00c02b60 XREF[0,1]: FUN_00518ab8:00518b38(R) | |
87 | 00c02b5e 47 3f 77 ds "G?www.auxbrain.com" | |
88 | 77 77 2e | |
89 | 61 75 78 | |
90 | s_HTTP_REQ:_%d_00c02b71 XREF[1]: makeRequestInternal:0067bbd4(*) | |
91 | 00c02b71 48 54 54 ds "HTTP REQ: %d" | |
92 | 50 20 52 | |
93 | 45 51 3a | |
94 | ``` | |
95 | Interesting, `www.auxbrain.com`. If we jump to its XREF, we get a garbled function, but what it seems to be doing is setting up | |
96 | certain global values. | |
97 | ||
98 | So we have a potential API endpoint, let's put it to the test. We're not going to recompile anything yet or do any byte-patching, | |
99 | let's try a quick smoke-test. Ensure your phone is rooted and you have a variant of Xposed Framework installed (I used LSPosed). | |
1e75f355 | 100 | We will need to unarm the SSL pinning present in most apps, including this one, I used [io.github.tehcneko.sslunpinning](https://github.com/Xposed-Modules-Repo/io.github.tehcneko.sslunpinning) module. |
2a5bb5a5 JM |
101 | (Note: I know it is possible to repackage the app to do SSL unpinning in most cases, but in many cases, you won't know if it's worth the effort yet) |
102 | ||
103 | Next, install [AdAway app from F-Droid](https://f-droid.org/packages/org.adaway/) so we can setup a redirection on any network we are on. | |
104 | Inside AdAway, add a redirection rule for the address we just found and point it to an IP address in your LAN that will run the API server. | |
105 | ||
106 | Generate a self-signed certificate authority and a certificate signed by it and run a webserver with both HTTP and HTTPS on the API server machine. | |
107 | Import the self-signed CA to your phone's truststore. Once all of that is done, run the app for first time. | |
108 | ||
109 | ``` | |
110 | 192.168.1.212 - - [...] "POST /ei/first_contact HTTP/1.1" 404 0 "-" | |
111 | ``` | |
112 | ||
113 | Bingo. We have contact and we have an API endpoint. Searching for "ei/" in the strings reveals a extensive list of API endpoints, we now have something | |
114 | to go off from. We have everything we need to start creating the server. | |
115 | ||
116 | ## Implementing the Server - Getting first contact | |
117 | Next up, we create a new project - as we generated the protobuf definitions for Python, we will proceed accordingly. | |
118 | If you are following along, get respective packages for your operating system to create python venvs. | |
119 | As the protobufs are being sent over HTTP, we will be serving our application over flask which is being reverse proxied by nginx. | |
120 | ||
121 | ``` | |
122 | # Lets stage the environment | |
123 | mkdir apiserver | |
124 | cd apiserver | |
125 | python -m venv .venv | |
126 | source .venv/bin/activate | |
127 | touch app.py | |
128 | cp ~/EggIncProtoExtractor/protos/ei.proto . | |
129 | ||
130 | # Get some dependeices | |
131 | pip install protobuf | |
132 | pip install flask | |
133 | ``` | |
134 | ||
135 | We now have the project set up for reading protobuf definitions and a framework to listen for HTTP and routes sent to it. | |
136 | Let's create an actual listener application, open app.py with your favourite IDE or text editor. | |
137 | ||
138 | ``` | |
139 | import ei_pb2 as EIProto | |
140 | from flask import Flask | |
141 | from flask import request | |
142 | ||
143 | @app.route("/ei/<path:subpath>", methods=["POST"]) | |
144 | def ei_routes(subpath): | |
145 | print("HTTP POST /ei/" + subpath) | |
146 | print(request.headers) | |
147 | return "" | |
148 | ``` | |
149 | ||
150 | This should get the ball rolling, we will see whatever call comes in and we can see what the payload of each request contains. | |
151 | At this point you should setup the reverse proxy, override your nginx / directive with: | |
152 | ``` | |
153 | location / { | |
154 | proxy_pass http://127.0.0.1:5000; | |
155 | } | |
156 | ``` | |
157 | ||
158 | Reload your nginx and start the flask application you just created with `flask run`. | |
159 | ||
160 | Run the app again and have it phone home and see what it contains. | |
161 | ``` | |
162 | HTTP POST /ei/first_contact | |
163 | Host: 127.0.0.1:5000 | |
164 | Connection: close | |
165 | Content-Length: 37 | |
166 | Content-Type: application/x-www-form-urlencoded | |
167 | User-Agent: Dalvik/2.1.0 (Linux; U; Android 13; M2012K11AG Build/TQ3A.230901.001) | |
168 | Accept-Encoding: gzip | |
169 | ``` | |
170 | ||
171 | We can see there's a form payload attached to this request, let's modify our app route a bit: | |
172 | ``` | |
173 | @app.route("/ei/<path:subpath>", methods=["POST"]) | |
174 | def ei_routes(subpath): | |
175 | print("HTTP POST /ei/" + subpath) | |
176 | print(request.form) | |
177 | return "" | |
178 | ``` | |
179 | ||
180 | Now if we run the modified flask application again, we see following output on the first_contact endpoint. | |
181 | ``` | |
182 | HTTP POST /ei/first_contact | |
183 | ImmutableMultiDict([('data', 'ChAzNTVlNDZlOTA4OWQxZTRjEAAYAg==')]) | |
184 | ``` | |
185 | ||
186 | We have a base64-encoded protobuf binary data - which isn't terribly useful for reading plain-text, since protobuf *is* a binary | |
187 | format, so we will need to figure out what protobuf message this payload belongs to. | |
188 | ||
189 | Remember that ei.proto file alongside the ei_pb2.py we got earlier? Lets go back there and inspect it a bit. | |
190 | We know we just contacted something called "first_contact", maybe there is something in that file that could help us? | |
191 | ||
192 | ``` | |
193 | message EggIncFirstContactRequest { | |
194 | optional string user_id = 1; | |
195 | optional uint32 client_version = 2; | |
196 | optional Platform platform = 3; | |
197 | } | |
198 | ||
199 | message EggIncFirstContactResponse { | |
200 | optional Backup backup = 1; | |
201 | } | |
202 | ``` | |
203 | ||
204 | Seems like the application is using message names in almost similar fashion to API endpoint names themselves. This will prove | |
205 | to be useful knowledge. We now know what the payload should be, lets put this to the test. | |
206 | ||
207 | Edit your app routine again | |
208 | ``` | |
209 | # add "import base64" to top of the file | |
210 | @app.route("/ei/<path:subpath>", methods=["POST"]) | |
211 | def ei_routes(subpath): | |
212 | print("HTTP POST /ei/" + subpath) | |
213 | if subpath == "first_contact": | |
214 | # Create the protobuf object so we can load data from the b64 payload | |
215 | FirstContact = EIProto.EggIncFirstContactRequest() | |
216 | FirstContact.ParseFromString(base64.b64decode(form["data"])) | |
217 | print(FirstContact) | |
218 | else: | |
219 | print(request.form) | |
220 | return "" | |
221 | ``` | |
222 | ||
223 | We should now be able to see deserialized output when we run the flask application and the mobile app, let's try it out: | |
224 | ||
225 | ``` | |
226 | HTTP POST /ei/first_contact | |
227 | user_id: "355e46e9089d1e4c" | |
228 | client_version: 0 | |
229 | platform: DROID | |
230 | ``` | |
231 | ||
232 | Nice! We now know how to identify which protobuf object corresponds to which API endpoint. We can now make an educated guess | |
233 | on what would come next. | |
234 | ||
235 | Seeing how we got `EggIncFirstContactRequest` and saw an adjacent `EggIncFirstContactResponse` message in the proto file, we | |
236 | can safely assume that this is what the game is expecting from us in return. | |
237 | ||
238 | Lets modify the server a bit to account for that. | |
239 | ||
240 | ``` | |
241 | @app.route("/ei/<path:subpath>", methods=["POST"]) | |
242 | def ei_routes(subpath): | |
243 | print("HTTP POST /ei/" + subpath) | |
244 | if subpath == "first_contact": | |
245 | # Create the protobuf object so we can load data from the b64 payload | |
246 | FirstContact = EIProto.EggIncFirstContactRequest() | |
247 | FirstContact.ParseFromString(base64.b64decode(form["data"])) | |
248 | print("We got a first contact hello from user " + FirstContact.user_id) | |
249 | # Lets respond with a FirstContactResponse | |
250 | FirstContactResp = EIProto.EggIncFirstContactResponse() | |
251 | # This takes only one optional argument - a Backup object - but we have no account | |
252 | # system yet, so we will opt out of sending that for now. | |
253 | # --- | |
254 | # We send the payload back as a base64 string - the same way we retrieved it. | |
255 | return base64.b64encode(FirstContactResp.SerializeToString()) | |
256 | else: | |
257 | print(request.form) | |
258 | return "" | |
259 | ``` | |
260 | ||
261 | Now when we run the app again, we notice that we no longer get spammed this endpoint, but instead in its place we see a few new friends | |
262 | ||
263 | ## Implementing the Server - New Friends | |
264 | Say hello to `/ei/save_backup` and `/ei/get_periodicals`. We can infer from the name, that save_backup would involve a Backup message | |
265 | and get_periodicals would involve a GetPeriodicalsRequest, both of which are defined fully in the proto spec file. | |
266 | ||
267 | Both of these are clogging up the flask application log periodically, we should check out what they are so we can have a sane log again. | |
268 | ||
269 | ``` | |
270 | @app.route("/ei/<path:subpath>", methods=["POST"]) | |
271 | def ei_routes(subpath): | |
272 | print("HTTP POST /ei/" + subpath) | |
273 | if subpath == "first_contact": | |
274 | # Create the protobuf object so we can load data from the b64 payload | |
275 | FirstContact = EIProto.EggIncFirstContactRequest() | |
276 | FirstContact.ParseFromString(base64.b64decode(form["data"])) | |
277 | print("We got a first contact hello from user " + FirstContact.user_id) | |
278 | # Lets respond with a FirstContactResponse | |
279 | FirstContactResp = EIProto.EggIncFirstContactResponse() | |
280 | # This takes only one optional argument - a Backup object - but we have no account | |
281 | # system yet, so we will opt out of sending that for now. | |
282 | # --- | |
283 | # We send the payload back as a base64 string - the same way we retrieved it. | |
284 | return base64.b64encode(FirstContactResp.SerializeToString()) | |
285 | elif subpath == "save_backup": | |
286 | # NOTE: This took me way longer to realize than it should have, but the base64 | |
287 | # payload you receive from client is broken due to some Android bug, where it | |
288 | # substitutes "+" symbols with a " " whitespace. | |
289 | # I don't want you to waste half hour to hours figuring out why you're getting | |
290 | # corrupted data, so you're welcome. | |
291 | Backup = EIProto.Backup() | |
292 | Backup.ParseFromString(base64.b64decode(form["data"].replace(" ", "+")) | |
293 | print(Backup) | |
294 | elif subpath == "get_periodicals": | |
295 | Periodicals = EIProto.GetPeriodicalsRequest() | |
296 | Periodicals.ParseFromString(base64.b64decode(form["data"]) | |
297 | print(Periodicals) | |
298 | else: | |
299 | print(request.form) | |
300 | return "" | |
301 | ``` | |
302 | ||
303 | We should now see what these payloads actually contain when deserialized (for your reading experience, I advise you to rather | |
304 | try this out yourself - the Backup structure is VERY large). | |
305 | ||
306 | Upon these payloads reaching the server, we see that a very much populated Backup message makes its way through and a relatively | |
307 | thin payload of PeriodicalsRequest comes through, which isn't very useful by itself. | |
308 | ||
309 | Now, this is why the game developer ended up creating forms of anticheat in future versions of this app - the Backup message contains | |
310 | your entire game state, which is often sent as a way to save your progress to cloud, but there is no actual sanity checking in place | |
311 | to ensure you're not progressing way too fast. Personally, I am of the mind that anticheat should be done on the server-side, not | |
312 | on the client-side, but I digress. We can use this to prove a very obvious vulnerability when using trust-client-always architecture. | |
313 | ||
314 | The game has an In App Purchase for "Pro Permit", which allows you to build more Silos, which in turn allow you to get offline | |
315 | rewards for a longer period of time. If we look at protobuf definition file, you can see under Backup.game, a field called "permit_level", | |
316 | which by default is zero. Lets try to change that and present a modified Backup the next time user opens the game. | |
317 | ||
318 | ||
319 | ``` | |
320 | cache = {} | |
321 | ||
322 | @app.route("/ei/<path:subpath>", methods=["POST"]) | |
323 | def ei_routes(subpath): | |
324 | print("HTTP POST /ei/" + subpath) | |
325 | if subpath == "first_contact": | |
326 | # Create the protobuf object so we can load data from the b64 payload | |
327 | FirstContact = EIProto.EggIncFirstContactRequest() | |
328 | FirstContact.ParseFromString(base64.b64decode(form["data"])) | |
329 | print("We got a first contact hello from user " + FirstContact.user_id) | |
330 | # Lets respond with a FirstContactResponse | |
331 | FirstContactResp = EIProto.EggIncFirstContactResponse() | |
332 | if FirstContact.user_id in cache: | |
333 | FirstContactResp.backup.CopyFrom(cache[FirstContact.user_id]) | |
334 | del cache[FirstContact.user_id] | |
335 | return base64.b64encode(FirstContactResp.SerializeToString()) | |
336 | elif subpath == "save_backup": | |
337 | # NOTE: This took me way longer to realize than it should have, but the base64 | |
338 | # payload you receive from client is broken due to some Android bug, where it | |
339 | # substitutes "+" symbols with a " " whitespace. | |
340 | # I don't want you to waste half hour to hours figuring out why you're getting | |
341 | # corrupted data, so you're welcome. | |
342 | Backup = EIProto.Backup() | |
343 | Backup.ParseFromString(base64.b64decode(form["data"].replace(" ", "+")) | |
344 | if Backup.game.permit_level == 0: | |
345 | print("Saved a modified Backup for next game load") | |
346 | # Modify the permit level, force offer the backup | |
347 | Backup.game.permit_level = 1 | |
348 | Backup.force_offer_backup = True | |
349 | Backup.force_backup = True | |
350 | cache[Backup.user_id] = Backup | |
351 | elif subpath == "get_periodicals": | |
352 | Periodicals = EIProto.GetPeriodicalsRequest() | |
353 | Periodicals.ParseFromString(base64.b64decode(form["data"]) | |
354 | print(Periodicals) | |
355 | else: | |
356 | print(request.form) | |
357 | return "" | |
358 | ``` | |
359 | ||
360 | Lets load up the game. Nothing interesting seems to be happening yet - lets wait until we see the "Saved a modified Backup for next game load" message | |
361 | show up in the server console. Once this shows up, restart the game - you are presented with a popup that you are offered to load a | |
362 | Backup from server. Let's accept that. | |
363 | ||
364 | Now click on your silos, you have the Pro Permit for free. | |
365 | ||
366 | Now, it goes without saying, I do not condone piracy - the future versions of this game are very much guarded against this, rightfully so. | |
367 | If you attempt this in actual game servers, this is considered fraud and IS detectable by the developer (every IAP has a receipt, logically!). | |
368 | ||
369 | This version of the game is defunct as the protocol has changed quite a bit in the years since this version and additional anticheat | |
370 | measures have been added since. You cannot transfer this status (or even purchase what you just did) from this game version to the next. | |
371 | ||
372 | ### Onto the PeriodicalsRequest | |
373 | This one is a bit more fun to delve into blindly - the proto spec wont help you much here. We will need to use our trusty RE tools again and delve into | |
374 | the game disassembly again. | |
375 | ||
376 | By public knowledge, we know there are server events for "Epic Research Sale", "Research Sale", "Drone Bonus" and "Prestige Boost". We can use this information to try | |
377 | and look at any potential leads in strings. Drone seems like a good canditate, lets look into that. | |
378 | ``` | |
379 | drone_fans2 | |
380 | drone_crash | |
381 | drone_enemy | |
382 | drone_hunter | |
383 | r_icon_drone_rewards | |
384 | b_icon_drone_boost | |
385 | drone_touch | |
386 | ei_drone_lights_green | |
387 | ei_drone_lights_red | |
388 | ei_drone_package | |
389 | ei_drone_propeller | |
390 | drone-boost | |
391 | GENEROUS DRONES | |
392 | ``` | |
393 | This looks promising, right off the bat, first strings I'd check here are `r_icon_drone_rewards`, `b_icon_drone_boost`, `drone-boost` and `GENEROUS DRONES`. | |
394 | ||
1e75f355 | 395 | I inspected all 4 of them, and when I got to the final 2, I found the enum string translations used for event IDs - here they are extracted for game version 1.12.13 |
2a5bb5a5 JM |
396 | ``` |
397 | piggy-boost (Rate piggy fills is increased.) | |
398 | piggy-cap-boost (UNLIMITED PIGGY;Gains are retained when event ends.) | |
399 | prestige-boost (PRESTIGE BOOST;Collect more soul eggs on prestige, you must prestige to take advantage of this event.) | |
400 | earnings-boost (CASH BOOST;Regular earnings are increased.) | |
401 | gift-boost (GENEROUS GIFTS;Boost applies to random gifts and video gifts.) | |
402 | drone-boost (GENEROUS DRONES;Drones will produce larger rewards.) | |
403 | epic-research-sale (EPIC RESEARCH SALE;Only applies to Epic Research.) | |
404 | vehicle-sale (VEHICLE SALE;Applies to all vehicles.) | |
405 | boost-sale (BOOST SALE;Applies to the gold price of boosts.) | |
406 | boost-duration (BOOST TIME+;Boosts last longer, you must start a boost during the event.) | |
407 | ``` | |
408 | I recall there being a few more boosts, but this is useful for getting started with compositing PeriodicalsResponse with an active running event. | |
409 | ||
410 | ### Putting together the response | |
411 | We have the enum, we have the names, descriptions, lets try to create a sample server event when the client enqueries about current server periodical events. | |
412 | ``` | |
413 | elif subpath == "get_periodicals": | |
414 | # We don't actually need the information client sends us, | |
415 | # we aren't verifying any stats about client in our server. | |
416 | CurrentPeriodicals = EIProto.PeriodicalsResponse() | |
417 | # In order to add items to a repeatable field in protobuf structure, | |
418 | # we need to call .add() method on it | |
419 | event = CurrentPeriodicals.events.events.add() | |
420 | # Refer to ei.proto - we are filling fields for EggIncEvent structure here. | |
421 | event.type = "drone-boost" | |
422 | event.multiplier = 5.00 | |
423 | event.subtitle = "Drones will produce larger rewards." | |
424 | event.identifier = "GENEROUS DRONES" | |
425 | event.seconds_remaining = 300.0 | |
426 | # Lets make it respond with a 5 minute event (this will re-arm itself when client calls | |
427 | # for get_periodicals again every 6 minutes) | |
428 | return base64.b64encode(CurrentPeriodicals.SerializeToString()) | |
429 | ``` | |
430 | ||
431 | Launch the server and observe as the client periodically calls this endpoint again, it will now receive a 5 minute 5x Drone Rewards boost on the game. | |
432 | ||
433 | ## Created the Server - What now? | |
434 | We have now created a very basic server, which appropriately responds to a first contact, misuses the game backup feature to prove a point about weaknesses of | |
435 | trusting client in server. We also created a very basic server event, which always rearms itself to never expire. | |
436 | ||
437 | What do we do next? | |
438 | ||
439 | At this point, we can start dog-fooding the project. Lets start with whatever ball game throws at us as we progress. | |
440 | ||
441 | ### Contracts | |
442 | As we progress the game and start performing prestiges, we unlock a feature called "Contracts" - but disaster strikes as we don't have any contracts we could | |
1e75f355 | 443 | accept. So far we still see our good friends `/ei/get_periodicals` and `/ei/save_backup` hammering the server at regular intervals. |
2a5bb5a5 JM |
444 | |
445 | When we created the periodicals response payload, you might have noticed in the protobuf message an optional field called `ContractsResponse contracts`. Lets see | |
446 | what this ContractsResponse message contains. | |
447 | ||
448 | ``` | |
449 | message ContractsResponse { | |
450 | repeated Contract contracts = 1; | |
451 | optional string warning_message = 4; | |
452 | optional double server_time = 2; | |
453 | optional uint32 max_eop = 3 [default = 1000]; | |
454 | } | |
455 | ``` | |
456 | ||
457 | Notice there being an array of Contract messages right off the bat - lets find its message structure next: | |
458 | ||
459 | ``` | |
460 | message Contract { | |
461 | optional string identifier = 1; | |
462 | optional string name = 9; | |
463 | optional string description = 10; | |
464 | optional Egg egg = 2; | |
465 | ||
466 | repeated Goal goals = 3; | |
467 | message Goal { | |
468 | optional GoalType type = 1; | |
469 | optional double target_amount = 2; | |
470 | optional RewardType reward_type = 3; | |
471 | optional string reward_sub_type = 4; | |
472 | optional double reward_amount = 5; | |
473 | optional double target_soul_eggs = 6; | |
474 | } | |
475 | ||
476 | repeated GoalSet goal_sets = 16; | |
477 | message GoalSet { | |
478 | repeated Goal goals = 1; | |
479 | } | |
480 | ||
481 | optional bool coop_allowed = 4; | |
482 | optional uint32 max_coop_size = 5; | |
483 | optional uint32 max_boosts = 12; | |
484 | optional double minutes_per_token = 15 [default = 60]; | |
485 | optional double expiration_time = 6; | |
486 | optional double length_seconds = 7; | |
487 | optional double max_soul_eggs = 13; | |
488 | optional uint32 min_client_version = 14; | |
489 | optional bool debug = 11; | |
490 | } | |
491 | ``` | |
492 | ||
493 | We will need to do a bit of reading. Fortunately, the game has a community wiki, lets look into how contracts should work. I took an | |
494 | [older revision of Contracts wiki page from 2021](https://egg-inc.fandom.com/wiki/Contracts?oldid=13015) and did some slight research. | |
495 | ||
496 | From what I gather, at one point, there was only one set of contract rewards, shared between everyone - then they created a system where | |
497 | beginners will get easier contract goals while more advanced players get harder contract goals. | |
498 | ||
499 | We can put two-and-two together here and infer that `repeated Goal goals` is the legacy contract system - where everyone was on equal footing | |
500 | and `repeated GoalSet goal_sets` is the *new* goal system that is split into Standard and Elite. | |
501 | ||
1e75f355 | 502 | We also learn that in future game versions, they completely reworked how contracts work *yet* again into a grading "bracket" system. Fortunately, |
2a5bb5a5 JM |
503 | we do not have to worry about that in our current target revision. |
504 | ||
505 | Now to get the ball rolling, there is conveniently a starting point set ahead for us already. The developer of game intended to ease new players into | |
506 | contracts by creating a simple & easy contract called [Your First Contract](https://egg-inc.fandom.com/wiki/Contracts/Your_First_Contract?oldid=13547). | |
507 | ||
508 | This page tells us all the information we need to compose our first Contract, so lets try to make one. | |
509 | ||
510 | ``` | |
511 | elif subpath == "get_periodicals": | |
512 | # We don't actually need the information client sends us, | |
513 | # we aren't verifying any stats about client in our server. | |
514 | CurrentPeriodicals = EIProto.PeriodicalsResponse() | |
515 | # [...] | |
516 | Contract = CurrentPeriodicals.contracts.contracts.add() | |
517 | Contract.identifier = "first-contract" | |
518 | Contract.name = "Your First Contract" | |
519 | Contract.description = "We heard you are open to contract work! Help fill this order from the local pharmacy!" | |
520 | Contract.egg = EIProto.Egg.MEDICAL | |
521 | Contract.coop_allowed = False | |
522 | Contract.minutes_per_token = 5 | |
523 | # Lets set expiry time to always be 3 days into future | |
524 | Contract.expiration_time = time.time() + (3600.0 * 72.0) | |
525 | Contract.length_seconds = 3600.0 * 4.0 | |
526 | # The wiki mentions that you cannot get this contract after you reach 5000 Soul Eggs | |
527 | Contract.max_soul_eggs = 5000.0 | |
528 | # We should have the basic metadata set now, lets create the goalsets. | |
529 | FirstSet = Contract.goal_sets.add() | |
530 | Goal = FirstSet.goals.add() | |
531 | # There is only one type of goal in this verison | |
532 | Goal.type = EIProto.GoalType.EGGS_LAID | |
533 | Goal.target_amount = 100000.0 | |
534 | Goal.reward_type = EIProto.RewardType.GOLD | |
535 | Goal.reward_amount = 192 | |
536 | Goal = FirstSet.goals.add() | |
537 | Goal.type = EIProto.GoalType.EGGS_LAID | |
538 | Goal.target_amount = 500000000.0 | |
539 | Goal.reward_type = EIProto.RewardType.PIGGY_FILL | |
540 | Goal.reward_amount = 10000 | |
541 | # Lets now add the Elite table, we can pretty much copy-paste the above here. | |
542 | SecondSet = Contract.goal_sets.add() | |
543 | Goal = SecondSet.goals.add() | |
544 | Goal.type = EIProto.GoalType.EGGS_LAID | |
545 | Goal.target_amount = 100000.0 | |
546 | Goal.reward_type = EIProto.RewardType.GOLD | |
547 | Goal.reward_amount = 500 | |
548 | Goal = SecondSet.goals.add() | |
549 | Goal.type = EIProto.GoalType.EGGS_LAID | |
550 | Goal.target_amount = 500000000.0 | |
551 | Goal.reward_type = EIProto.RewardType.PIGGY_FILL | |
552 | Goal.reward_amount = 10000 | |
553 | return base64.b64encode(CurrentPeriodicals.SerializeToString()) | |
554 | ``` | |
555 | ||
1e75f355 | 556 | Lets try that out in-game now - after waiting for a minute, we see our contract pop up, but I immediately noticed one thing amiss. |
2a5bb5a5 JM |
557 | The contract goals are swapped! I am getting Elite contract rewards for a Standard contract. |
558 | ||
559 | This piece of information now tells us that the first entry in GoalSets refers to Elite rewards and the second entry in GoalSets to Standard rewards. | |
560 | After swapping the sets around, we now see a contract with the corrected rewards. | |
561 | ||
562 | I playtested it a bit and the contract worked as expected. | |
563 | ||
564 | Now, the above code could be a lot neater. For your homework, if you're not skipping to the public source release in the end, you should try to create | |
565 | a contract database and try scheduling them like the game originally did - a "Leggacy" contract every Friday and regular contracts showing up every 1-2 weeks | |
566 | for roughly 2 weeks. | |
567 | ||
1e75f355 JM |
568 | ## Rootless SSL Unpinning + Endpoint URL patching |
569 | Let's make the app not require a VPN or root privileges - let's make user CAs work and the endpoint URL something we control on the public net. | |
570 | Start off by pulling the following repository | |
571 | ``` | |
572 | git clone https://github.com/ilya-kozyr/android-ssl-pinning-bypass.git | |
573 | python3 -m venv .venv | |
574 | source .venv/bin/activate | |
575 | pip install -r requirements.txt | |
576 | cp /path/to/your/apk . | |
577 | python3 apk-rebuild.py egginc.apk --pause | |
578 | ``` | |
579 | ||
580 | Open a new terminal window, the script will wait for us to perform modifications, enter the created folder `egginc.apk-decompiled` and `lib`. | |
581 | ||
582 | We have two folders here now, `arm64-v8a` and `armeabi-v7a`, just as we saw when we pulled the .so file out of the apk earlier. Let's tackle | |
583 | the 64-bit build first. | |
584 | ||
585 | For arm64 build it was really simple to perform bytepatch on the said endpoint. We already know it's supposed to look as `G?www.auxbrain.com` - let's probe the .so library a bit. | |
586 | ``` | |
587 | $> hexdump -C libegginc.so | grep "ww.auxbrain.co" -A2 -B2 | |
588 | 00b02b40 cd cc 4c 3f 00 00 00 00 00 00 00 00 00 00 80 3f |..L?...........?| | |
589 | 00b02b50 00 00 00 00 00 00 00 00 00 00 00 00 14 ae 47 3f |..............G?| | |
590 | 00b02b60 77 77 77 2e 61 75 78 62 72 61 69 6e 2e 63 6f 6d |www.auxbrain.com| | |
591 | 00b02b70 00 48 54 54 50 20 52 45 51 3a 20 25 64 00 64 61 |.HTTP REQ: %d.da| | |
592 | 00b02b80 74 61 3d 00 65 69 2f 66 69 72 73 74 5f 63 6f 6e |ta=.ei/first_con | |
593 | ``` | |
594 | ||
595 | We seem to have nothing blocking our way, let's create hex representations of `G?www.auxbrain.com` and a target domain of equal length, for example `G?eggs.based.quest`. | |
596 | ||
597 | (Note: You can choose a shorter name as well, if you null-terminate the extra bytes as padding) | |
598 | ``` | |
599 | $> echo "G?www.auxbrain.com" | hexdump -ve '1/1 "%.2X"' | |
600 | 473F7777772E617578627261696E2E636F6D0A | |
601 | $> echo "G?eggs.based.quest" | hexdump -ve '1/1 "%.2X"' | |
602 | 473F656767732E62617365642E71756573740A | |
603 | ``` | |
604 | ||
605 | Remove the trailing `0A` from end of both hex strings and now proceed as follows: | |
606 | ``` | |
607 | # Place the source in first bracket of sed and the new URL at second bracket. | |
608 | hexdump -ve '1/1 "%.2X"' libegginc.so | sed "s/473F7777772E617578627261696E2E636F6D/473F656767732E62617365642E7175657374/g" | xxd -r -p > patched.so | |
609 | ``` | |
610 | ||
611 | Huzzah! We now have a patched linked-library for the arm64 build. Let's also patch the 32-bit version. | |
612 | ``` | |
613 | $> hexdump -C libegginc.so | grep "ww.auxbrain.co" -A2 -B2 | |
614 | 0087b770 69 67 68 5f 74 6f 6f 5f 6d 61 6e 79 5f 70 78 00 |igh_too_many_px.| | |
615 | 0087b780 74 61 62 6c 65 74 5f 68 64 70 69 00 00 00 00 00 |tablet_hdpi.....| | |
616 | 0087b790 77 77 77 2e 61 75 78 62 72 61 69 6e 2e 63 6f 6d |www.auxbrain.com| | |
617 | 0087b7a0 00 00 00 00 00 00 00 00 65 69 2f 66 69 72 73 74 |........ei/first| | |
618 | 0087b7b0 5f 63 6f 6e 74 61 63 74 00 00 00 00 00 00 00 00 |_contact........| | |
619 | ``` | |
620 | This one lacks the `G?` prefix on API endpoint, but we still have null terminators we can rely on. Let's replace the `473F` from our previous strings with `0000`. | |
621 | ``` | |
622 | # Place the source in first bracket of sed and the new URL at second bracket. | |
623 | hexdump -ve '1/1 "%.2X"' libegginc.so | sed "s/00007777772E617578627261696E2E636F6D/0000656767732E62617365642E7175657374/g" | xxd -r -p > patched.so | |
624 | ``` | |
625 | ||
626 | Replace both of the libegginc.so files with the patched.so files. Move back to main terminal window and press ENTER. | |
627 | ||
628 | We now have a patched and debug signed apk for the game that isn't SSL pinned and contains a custom API endpoint we control without a VPN. | |
629 | ||
2a5bb5a5 JM |
630 | ## Conclusion so far |
631 | We have created a (rather ugly looking) server emulator for the game. It functions, but it needs a lot of work still before we can call it ready. | |
632 | If you have followed this far, give yourself pat on the back - if you actually tried to run this code, give yourself an extra pat on the back. | |
633 | ||
634 | Before I give you the public source to the project, you might want to try your hand at creating a few more things. | |
635 | - "Cloud" save, present a Backup to any new device that just started playing. | |
636 | - Contracts Database and scheduler | |
637 | - Server Event scheduler | |
638 | ||
639 | I apologize if my method of documenting this has been messy, but that's also part of the chaos of reverse engineering, you are constantly learning new things | |
640 | about the project you are currently doing - refactoring becomes an essential part once you have documented the protocol to a comfortable degree. | |
641 | ||
642 | I won't give any promises for a part 2 any time soon, but I will be trying to make this feature complete, so without further ado, here are the git repository links: | |
643 | [github.com](https://github.com/cernodile/reEgg), [git.based.quest](https://git.based.quest/?p=reEgg.git;a=tree;h=refs/heads/master;hb=refs/heads/master). | |
644 | ||
645 | Thank you for reading and making it all the way to the end, | |
646 | - Cernodile |