Commit | Line | Data |
---|---|---|
2a5bb5a5 JM |
1 | --- |
2 | title: "Reverse Engineering a Mobile App Protobuf API" | |
3 | date: 2024-05-11T12:00:00+03:00 | |
4 | description: "In recent times more than ever, live service and overuse of APIs soon-to-be-stale is only increasing. This causes a lot of once-written software to become either unusable or handicapped in many regards. One way to fight this is to learn reverse engineering for sake of digital preservation. In this blog, I take you through a journey of reverse engineering a simple mobile game's protobuf API." | |
5 | tags: ['tutorial', 'reverse-engineering', 'opensource'] | |
6 | type: blog | |
7 | draft: false | |
8 | --- | |
9 | ||
10 | # Why | |
11 | Why not? Digital preservation is important, even if you don't care for a specific program. | |
12 | This is also a good way to get started with protocol reverse engineering due to way protobuf | |
13 | is often left behind in source format within client applications. | |
14 | ||
15 | ## The target | |
16 | In this series of blogposts, I will be using a mobile game "Egg, Inc." as the target for | |
17 | demonstration. It's a simple time killer app that got me through boring long waits when I was still at school. | |
18 | ||
1e75f355 JM |
19 | Egg, Inc. is a basic incremental "idler" game where your goal is to take over the world food supply with ever-increasing supply of eggs, |
20 | if you have ever played Cookie Clicker, you know the premise of something like that. You have to unlock denser and denser eggs - the game | |
21 | is also designed around the fact that you can do certain online-tied activites such as Contracts to unlock more Soul Eggs (prestige boost) and | |
22 | "Eggs of Prophecy" which increase potency of your Soul Eggs. | |
23 | ||
24 | It's rather simple game with a very minimal API, making it perfect for learning. You may not like the game, but that's beside the point. | |
25 | The simplicity of our target matters here. | |
2a5bb5a5 JM |
26 | |
27 | ## The existing works | |
1e75f355 JM |
28 | In some cases, you will find previous works on the target you pick. In my case, some clever people have created |
29 | [scripts to extract .proto file out of the app.](https://github.com/DavidArthurCole/EggIncProtoExtractor) | |
2a5bb5a5 JM |
30 | I advise you to check it out if you wish to get a better understanding of how you would go about retrieving the |
31 | API spec .proto file for your target. | |
32 | ||
33 | Further there are a few dedicated individuals in the game's community who have created numerous tools and historical databases. | |
34 | ||
35 | For this blog purposes, we will assume the game server is shut down (as in we cannot query from the live API) and our goal is | |
36 | to make a semi-functional selfhosted gameserver for our own needs, assuming we are the only one on said server. | |
37 | ||
1e75f355 JM |
38 | ## How to source builds of a game |
39 | There are two methods of sourcing the apk file here - one method is if you already have the app installed, install something like ZArchiver | |
40 | and extract it from /data/app/ - identifying the app by its icon. From there you will find `base.apk` which is enough for most apps. | |
41 | ||
42 | Alternatively, if the app is still available on Google Play, you can use an app like Aurora Store to go to the store detail page, select | |
43 | "Manual Download" and enter a known Build ID. | |
44 | ||
2a5bb5a5 JM |
45 | ## Getting Started |
46 | Thanks to the previously mentioned script, it's easy to get started - find the APK, extract protobuf spec file, convert it with | |
47 | protoc and we're done there. One small problem - due to cheaters, latest version of the game includes "AuthenticatedMessage" structure, | |
48 | which contains a salted sha256sum of the payload message. | |
49 | ||
50 | At this point, after a bit of internal dilemma, I decided to not further the problem while service is still live for people playing and did the | |
51 | more morally sound decision of picking a version prior to these integrity checks. We can crack that another day as all the needed information | |
52 | is retained in the app itself. | |
53 | ||
1e75f355 | 54 | Going forward with this, we are targetting game version 1.12.13 (Build ID 111121 - use that in Aurora Store). |
2a5bb5a5 JM |
55 | |
56 | With all that out of the way, lets get into actual commands used here: | |
57 | ``` | |
58 | git clone https://github.com/DavidArthurCole/EggIncProtoExtractor.git | |
59 | cd EggIncProtoExtractor | |
60 | ./apkextract.sh com.auxbrain.egginc_1.12.13.apk | |
61 | # We should have a new folder "protos" now with resulting files | |
62 | cd protos | |
63 | # There should be a file called ei.proto - that's our protobuf spec file | |
64 | # At this point, we can use the protoc utility which can convert the specfile | |
65 | # to interfaces in C++, C#, Java, Kotlin, Objective-C, PHP, Python and Ruby with | |
66 | # additional plugin support for Dart and Go. | |
67 | # To make this easier to understand, we will use Python in this demonstration | |
68 | protoc -I=. --python_out=. ./ei.proto | |
69 | # Success! We now have a "ei_pb2.py" file which can be directly imported to Python programs | |
70 | ``` | |
71 | ||
72 | With the protobuf interface in Python created, we can now proceed with creating the API emulator - but there's a slight problem. | |
73 | What URL? What endpoints? How do we find this out? Simple answer, disassembling the game. Get your RE tool of choice, I will be | |
74 | using [Ghidra](https://ghidra-sre.org/) myself. | |
75 | ||
76 | (Note: You can also just try to find this out using tools such as WireShark) | |
77 | ||
78 | The game contains a linked-library written in C++, which you can find inside the .apk `lib` folder, named as `libegginc.so`. | |
79 | This is perfect for our use-case, Ghidra is going to slice through this like butter. Import the file to your RE tool of choice | |
80 | and let it perform some analysis on it, have a cup of tea or coffee as this is going to take a hot minute. | |
81 | ||
82 | Once that's done, we are going to start by looking at the defined strings - try our luck there. Search for any debug prints left | |
83 | behind or maybe some clues. I started by searching for `http`, which lead me to following string `"HTTP REQ: %d"`, seems promising. | |
84 | When I jumped to it, I saw an exactly adjacent string to it which could give more clues: | |
85 | ``` | |
86 | s_www.auxbrain.com_00c02b60 XREF[0,1]: FUN_00518ab8:00518b38(R) | |
87 | 00c02b5e 47 3f 77 ds "G?www.auxbrain.com" | |
88 | 77 77 2e | |
89 | 61 75 78 | |
90 | s_HTTP_REQ:_%d_00c02b71 XREF[1]: makeRequestInternal:0067bbd4(*) | |
91 | 00c02b71 48 54 54 ds "HTTP REQ: %d" | |
92 | 50 20 52 | |
93 | 45 51 3a | |
94 | ``` | |
95 | Interesting, `www.auxbrain.com`. If we jump to its XREF, we get a garbled function, but what it seems to be doing is setting up | |
96 | certain global values. | |
97 | ||
be8f6a91 JM |
98 | ## The smoke-test |
99 | ||
72731d2b | 100 | So we have a potential API endpoint, let's put it to the test. We can do a quick smoke test by setting up a webserver. |
37ff2de8 | 101 | |
72731d2b | 102 | Install [AdAway app from F-Droid](https://f-droid.org/packages/org.adaway/) so we can setup a redirection on any network we are on. |
2a5bb5a5 JM |
103 | Inside AdAway, add a redirection rule for the address we just found and point it to an IP address in your LAN that will run the API server. |
104 | ||
72731d2b | 105 | (NOTE: AdAway doesn't detect any subdomains nor can it do wildcard, you will need to include the FQDN of the API endpoint `www.auxbrain.com`) |
2a5bb5a5 | 106 | |
72731d2b | 107 | Once you're done setting up the redirection, run any webserver such as nginx for a quick and dirty test. |
2a5bb5a5 JM |
108 | ``` |
109 | 192.168.1.212 - - [...] "POST /ei/first_contact HTTP/1.1" 404 0 "-" | |
110 | ``` | |
111 | ||
112 | Bingo. We have contact and we have an API endpoint. Searching for "ei/" in the strings reveals a extensive list of API endpoints, we now have something | |
113 | to go off from. We have everything we need to start creating the server. | |
114 | ||
115 | ## Implementing the Server - Getting first contact | |
116 | Next up, we create a new project - as we generated the protobuf definitions for Python, we will proceed accordingly. | |
117 | If you are following along, get respective packages for your operating system to create python venvs. | |
118 | As the protobufs are being sent over HTTP, we will be serving our application over flask which is being reverse proxied by nginx. | |
119 | ||
120 | ``` | |
121 | # Lets stage the environment | |
122 | mkdir apiserver | |
123 | cd apiserver | |
124 | python -m venv .venv | |
125 | source .venv/bin/activate | |
126 | touch app.py | |
127 | cp ~/EggIncProtoExtractor/protos/ei.proto . | |
128 | ||
129 | # Get some dependeices | |
130 | pip install protobuf | |
131 | pip install flask | |
132 | ``` | |
133 | ||
134 | We now have the project set up for reading protobuf definitions and a framework to listen for HTTP and routes sent to it. | |
135 | Let's create an actual listener application, open app.py with your favourite IDE or text editor. | |
136 | ||
137 | ``` | |
138 | import ei_pb2 as EIProto | |
139 | from flask import Flask | |
140 | from flask import request | |
141 | ||
142 | @app.route("/ei/<path:subpath>", methods=["POST"]) | |
143 | def ei_routes(subpath): | |
144 | print("HTTP POST /ei/" + subpath) | |
145 | print(request.headers) | |
146 | return "" | |
147 | ``` | |
148 | ||
149 | This should get the ball rolling, we will see whatever call comes in and we can see what the payload of each request contains. | |
150 | At this point you should setup the reverse proxy, override your nginx / directive with: | |
151 | ``` | |
152 | location / { | |
153 | proxy_pass http://127.0.0.1:5000; | |
154 | } | |
155 | ``` | |
156 | ||
157 | Reload your nginx and start the flask application you just created with `flask run`. | |
158 | ||
159 | Run the app again and have it phone home and see what it contains. | |
160 | ``` | |
161 | HTTP POST /ei/first_contact | |
162 | Host: 127.0.0.1:5000 | |
163 | Connection: close | |
164 | Content-Length: 37 | |
165 | Content-Type: application/x-www-form-urlencoded | |
166 | User-Agent: Dalvik/2.1.0 (Linux; U; Android 13; M2012K11AG Build/TQ3A.230901.001) | |
167 | Accept-Encoding: gzip | |
168 | ``` | |
169 | ||
170 | We can see there's a form payload attached to this request, let's modify our app route a bit: | |
171 | ``` | |
172 | @app.route("/ei/<path:subpath>", methods=["POST"]) | |
173 | def ei_routes(subpath): | |
174 | print("HTTP POST /ei/" + subpath) | |
175 | print(request.form) | |
176 | return "" | |
177 | ``` | |
178 | ||
179 | Now if we run the modified flask application again, we see following output on the first_contact endpoint. | |
180 | ``` | |
181 | HTTP POST /ei/first_contact | |
182 | ImmutableMultiDict([('data', 'ChAzNTVlNDZlOTA4OWQxZTRjEAAYAg==')]) | |
183 | ``` | |
184 | ||
185 | We have a base64-encoded protobuf binary data - which isn't terribly useful for reading plain-text, since protobuf *is* a binary | |
186 | format, so we will need to figure out what protobuf message this payload belongs to. | |
187 | ||
188 | Remember that ei.proto file alongside the ei_pb2.py we got earlier? Lets go back there and inspect it a bit. | |
189 | We know we just contacted something called "first_contact", maybe there is something in that file that could help us? | |
190 | ||
191 | ``` | |
192 | message EggIncFirstContactRequest { | |
193 | optional string user_id = 1; | |
194 | optional uint32 client_version = 2; | |
195 | optional Platform platform = 3; | |
196 | } | |
197 | ||
198 | message EggIncFirstContactResponse { | |
199 | optional Backup backup = 1; | |
200 | } | |
201 | ``` | |
202 | ||
203 | Seems like the application is using message names in almost similar fashion to API endpoint names themselves. This will prove | |
204 | to be useful knowledge. We now know what the payload should be, lets put this to the test. | |
205 | ||
206 | Edit your app routine again | |
207 | ``` | |
208 | # add "import base64" to top of the file | |
209 | @app.route("/ei/<path:subpath>", methods=["POST"]) | |
210 | def ei_routes(subpath): | |
211 | print("HTTP POST /ei/" + subpath) | |
212 | if subpath == "first_contact": | |
213 | # Create the protobuf object so we can load data from the b64 payload | |
214 | FirstContact = EIProto.EggIncFirstContactRequest() | |
215 | FirstContact.ParseFromString(base64.b64decode(form["data"])) | |
216 | print(FirstContact) | |
217 | else: | |
218 | print(request.form) | |
219 | return "" | |
220 | ``` | |
221 | ||
222 | We should now be able to see deserialized output when we run the flask application and the mobile app, let's try it out: | |
223 | ||
224 | ``` | |
225 | HTTP POST /ei/first_contact | |
226 | user_id: "355e46e9089d1e4c" | |
227 | client_version: 0 | |
228 | platform: DROID | |
229 | ``` | |
230 | ||
231 | Nice! We now know how to identify which protobuf object corresponds to which API endpoint. We can now make an educated guess | |
232 | on what would come next. | |
233 | ||
234 | Seeing how we got `EggIncFirstContactRequest` and saw an adjacent `EggIncFirstContactResponse` message in the proto file, we | |
235 | can safely assume that this is what the game is expecting from us in return. | |
236 | ||
237 | Lets modify the server a bit to account for that. | |
238 | ||
239 | ``` | |
240 | @app.route("/ei/<path:subpath>", methods=["POST"]) | |
241 | def ei_routes(subpath): | |
242 | print("HTTP POST /ei/" + subpath) | |
243 | if subpath == "first_contact": | |
244 | # Create the protobuf object so we can load data from the b64 payload | |
245 | FirstContact = EIProto.EggIncFirstContactRequest() | |
246 | FirstContact.ParseFromString(base64.b64decode(form["data"])) | |
247 | print("We got a first contact hello from user " + FirstContact.user_id) | |
248 | # Lets respond with a FirstContactResponse | |
249 | FirstContactResp = EIProto.EggIncFirstContactResponse() | |
250 | # This takes only one optional argument - a Backup object - but we have no account | |
251 | # system yet, so we will opt out of sending that for now. | |
252 | # --- | |
253 | # We send the payload back as a base64 string - the same way we retrieved it. | |
254 | return base64.b64encode(FirstContactResp.SerializeToString()) | |
255 | else: | |
256 | print(request.form) | |
257 | return "" | |
258 | ``` | |
259 | ||
260 | Now when we run the app again, we notice that we no longer get spammed this endpoint, but instead in its place we see a few new friends | |
261 | ||
262 | ## Implementing the Server - New Friends | |
263 | Say hello to `/ei/save_backup` and `/ei/get_periodicals`. We can infer from the name, that save_backup would involve a Backup message | |
264 | and get_periodicals would involve a GetPeriodicalsRequest, both of which are defined fully in the proto spec file. | |
265 | ||
266 | Both of these are clogging up the flask application log periodically, we should check out what they are so we can have a sane log again. | |
267 | ||
268 | ``` | |
269 | @app.route("/ei/<path:subpath>", methods=["POST"]) | |
270 | def ei_routes(subpath): | |
271 | print("HTTP POST /ei/" + subpath) | |
272 | if subpath == "first_contact": | |
273 | # Create the protobuf object so we can load data from the b64 payload | |
274 | FirstContact = EIProto.EggIncFirstContactRequest() | |
275 | FirstContact.ParseFromString(base64.b64decode(form["data"])) | |
276 | print("We got a first contact hello from user " + FirstContact.user_id) | |
277 | # Lets respond with a FirstContactResponse | |
278 | FirstContactResp = EIProto.EggIncFirstContactResponse() | |
279 | # This takes only one optional argument - a Backup object - but we have no account | |
280 | # system yet, so we will opt out of sending that for now. | |
281 | # --- | |
282 | # We send the payload back as a base64 string - the same way we retrieved it. | |
283 | return base64.b64encode(FirstContactResp.SerializeToString()) | |
284 | elif subpath == "save_backup": | |
285 | # NOTE: This took me way longer to realize than it should have, but the base64 | |
286 | # payload you receive from client is broken due to some Android bug, where it | |
287 | # substitutes "+" symbols with a " " whitespace. | |
288 | # I don't want you to waste half hour to hours figuring out why you're getting | |
289 | # corrupted data, so you're welcome. | |
290 | Backup = EIProto.Backup() | |
291 | Backup.ParseFromString(base64.b64decode(form["data"].replace(" ", "+")) | |
292 | print(Backup) | |
293 | elif subpath == "get_periodicals": | |
294 | Periodicals = EIProto.GetPeriodicalsRequest() | |
295 | Periodicals.ParseFromString(base64.b64decode(form["data"]) | |
296 | print(Periodicals) | |
297 | else: | |
298 | print(request.form) | |
299 | return "" | |
300 | ``` | |
301 | ||
302 | We should now see what these payloads actually contain when deserialized (for your reading experience, I advise you to rather | |
303 | try this out yourself - the Backup structure is VERY large). | |
304 | ||
305 | Upon these payloads reaching the server, we see that a very much populated Backup message makes its way through and a relatively | |
306 | thin payload of PeriodicalsRequest comes through, which isn't very useful by itself. | |
307 | ||
308 | Now, this is why the game developer ended up creating forms of anticheat in future versions of this app - the Backup message contains | |
309 | your entire game state, which is often sent as a way to save your progress to cloud, but there is no actual sanity checking in place | |
310 | to ensure you're not progressing way too fast. Personally, I am of the mind that anticheat should be done on the server-side, not | |
311 | on the client-side, but I digress. We can use this to prove a very obvious vulnerability when using trust-client-always architecture. | |
312 | ||
313 | The game has an In App Purchase for "Pro Permit", which allows you to build more Silos, which in turn allow you to get offline | |
314 | rewards for a longer period of time. If we look at protobuf definition file, you can see under Backup.game, a field called "permit_level", | |
315 | which by default is zero. Lets try to change that and present a modified Backup the next time user opens the game. | |
316 | ||
317 | ||
318 | ``` | |
319 | cache = {} | |
320 | ||
321 | @app.route("/ei/<path:subpath>", methods=["POST"]) | |
322 | def ei_routes(subpath): | |
323 | print("HTTP POST /ei/" + subpath) | |
324 | if subpath == "first_contact": | |
325 | # Create the protobuf object so we can load data from the b64 payload | |
326 | FirstContact = EIProto.EggIncFirstContactRequest() | |
327 | FirstContact.ParseFromString(base64.b64decode(form["data"])) | |
328 | print("We got a first contact hello from user " + FirstContact.user_id) | |
329 | # Lets respond with a FirstContactResponse | |
330 | FirstContactResp = EIProto.EggIncFirstContactResponse() | |
331 | if FirstContact.user_id in cache: | |
332 | FirstContactResp.backup.CopyFrom(cache[FirstContact.user_id]) | |
333 | del cache[FirstContact.user_id] | |
334 | return base64.b64encode(FirstContactResp.SerializeToString()) | |
335 | elif subpath == "save_backup": | |
336 | # NOTE: This took me way longer to realize than it should have, but the base64 | |
337 | # payload you receive from client is broken due to some Android bug, where it | |
338 | # substitutes "+" symbols with a " " whitespace. | |
339 | # I don't want you to waste half hour to hours figuring out why you're getting | |
340 | # corrupted data, so you're welcome. | |
341 | Backup = EIProto.Backup() | |
342 | Backup.ParseFromString(base64.b64decode(form["data"].replace(" ", "+")) | |
343 | if Backup.game.permit_level == 0: | |
344 | print("Saved a modified Backup for next game load") | |
345 | # Modify the permit level, force offer the backup | |
346 | Backup.game.permit_level = 1 | |
347 | Backup.force_offer_backup = True | |
348 | Backup.force_backup = True | |
349 | cache[Backup.user_id] = Backup | |
350 | elif subpath == "get_periodicals": | |
351 | Periodicals = EIProto.GetPeriodicalsRequest() | |
352 | Periodicals.ParseFromString(base64.b64decode(form["data"]) | |
353 | print(Periodicals) | |
354 | else: | |
355 | print(request.form) | |
356 | return "" | |
357 | ``` | |
358 | ||
359 | Lets load up the game. Nothing interesting seems to be happening yet - lets wait until we see the "Saved a modified Backup for next game load" message | |
360 | show up in the server console. Once this shows up, restart the game - you are presented with a popup that you are offered to load a | |
361 | Backup from server. Let's accept that. | |
362 | ||
363 | Now click on your silos, you have the Pro Permit for free. | |
364 | ||
365 | Now, it goes without saying, I do not condone piracy - the future versions of this game are very much guarded against this, rightfully so. | |
366 | If you attempt this in actual game servers, this is considered fraud and IS detectable by the developer (every IAP has a receipt, logically!). | |
367 | ||
368 | This version of the game is defunct as the protocol has changed quite a bit in the years since this version and additional anticheat | |
369 | measures have been added since. You cannot transfer this status (or even purchase what you just did) from this game version to the next. | |
370 | ||
371 | ### Onto the PeriodicalsRequest | |
372 | This one is a bit more fun to delve into blindly - the proto spec wont help you much here. We will need to use our trusty RE tools again and delve into | |
373 | the game disassembly again. | |
374 | ||
375 | By public knowledge, we know there are server events for "Epic Research Sale", "Research Sale", "Drone Bonus" and "Prestige Boost". We can use this information to try | |
376 | and look at any potential leads in strings. Drone seems like a good canditate, lets look into that. | |
377 | ``` | |
378 | drone_fans2 | |
379 | drone_crash | |
380 | drone_enemy | |
381 | drone_hunter | |
382 | r_icon_drone_rewards | |
383 | b_icon_drone_boost | |
384 | drone_touch | |
385 | ei_drone_lights_green | |
386 | ei_drone_lights_red | |
387 | ei_drone_package | |
388 | ei_drone_propeller | |
389 | drone-boost | |
390 | GENEROUS DRONES | |
391 | ``` | |
392 | This looks promising, right off the bat, first strings I'd check here are `r_icon_drone_rewards`, `b_icon_drone_boost`, `drone-boost` and `GENEROUS DRONES`. | |
393 | ||
1e75f355 | 394 | I inspected all 4 of them, and when I got to the final 2, I found the enum string translations used for event IDs - here they are extracted for game version 1.12.13 |
2a5bb5a5 JM |
395 | ``` |
396 | piggy-boost (Rate piggy fills is increased.) | |
397 | piggy-cap-boost (UNLIMITED PIGGY;Gains are retained when event ends.) | |
398 | prestige-boost (PRESTIGE BOOST;Collect more soul eggs on prestige, you must prestige to take advantage of this event.) | |
399 | earnings-boost (CASH BOOST;Regular earnings are increased.) | |
400 | gift-boost (GENEROUS GIFTS;Boost applies to random gifts and video gifts.) | |
401 | drone-boost (GENEROUS DRONES;Drones will produce larger rewards.) | |
402 | epic-research-sale (EPIC RESEARCH SALE;Only applies to Epic Research.) | |
403 | vehicle-sale (VEHICLE SALE;Applies to all vehicles.) | |
404 | boost-sale (BOOST SALE;Applies to the gold price of boosts.) | |
405 | boost-duration (BOOST TIME+;Boosts last longer, you must start a boost during the event.) | |
406 | ``` | |
407 | I recall there being a few more boosts, but this is useful for getting started with compositing PeriodicalsResponse with an active running event. | |
408 | ||
409 | ### Putting together the response | |
410 | We have the enum, we have the names, descriptions, lets try to create a sample server event when the client enqueries about current server periodical events. | |
411 | ``` | |
412 | elif subpath == "get_periodicals": | |
413 | # We don't actually need the information client sends us, | |
414 | # we aren't verifying any stats about client in our server. | |
415 | CurrentPeriodicals = EIProto.PeriodicalsResponse() | |
416 | # In order to add items to a repeatable field in protobuf structure, | |
417 | # we need to call .add() method on it | |
418 | event = CurrentPeriodicals.events.events.add() | |
419 | # Refer to ei.proto - we are filling fields for EggIncEvent structure here. | |
420 | event.type = "drone-boost" | |
421 | event.multiplier = 5.00 | |
422 | event.subtitle = "Drones will produce larger rewards." | |
423 | event.identifier = "GENEROUS DRONES" | |
424 | event.seconds_remaining = 300.0 | |
425 | # Lets make it respond with a 5 minute event (this will re-arm itself when client calls | |
426 | # for get_periodicals again every 6 minutes) | |
427 | return base64.b64encode(CurrentPeriodicals.SerializeToString()) | |
428 | ``` | |
429 | ||
430 | Launch the server and observe as the client periodically calls this endpoint again, it will now receive a 5 minute 5x Drone Rewards boost on the game. | |
431 | ||
432 | ## Created the Server - What now? | |
433 | We have now created a very basic server, which appropriately responds to a first contact, misuses the game backup feature to prove a point about weaknesses of | |
434 | trusting client in server. We also created a very basic server event, which always rearms itself to never expire. | |
435 | ||
436 | What do we do next? | |
437 | ||
438 | At this point, we can start dog-fooding the project. Lets start with whatever ball game throws at us as we progress. | |
439 | ||
440 | ### Contracts | |
441 | As we progress the game and start performing prestiges, we unlock a feature called "Contracts" - but disaster strikes as we don't have any contracts we could | |
1e75f355 | 442 | accept. So far we still see our good friends `/ei/get_periodicals` and `/ei/save_backup` hammering the server at regular intervals. |
2a5bb5a5 JM |
443 | |
444 | When we created the periodicals response payload, you might have noticed in the protobuf message an optional field called `ContractsResponse contracts`. Lets see | |
445 | what this ContractsResponse message contains. | |
446 | ||
447 | ``` | |
448 | message ContractsResponse { | |
449 | repeated Contract contracts = 1; | |
450 | optional string warning_message = 4; | |
451 | optional double server_time = 2; | |
452 | optional uint32 max_eop = 3 [default = 1000]; | |
453 | } | |
454 | ``` | |
455 | ||
456 | Notice there being an array of Contract messages right off the bat - lets find its message structure next: | |
457 | ||
458 | ``` | |
459 | message Contract { | |
460 | optional string identifier = 1; | |
461 | optional string name = 9; | |
462 | optional string description = 10; | |
463 | optional Egg egg = 2; | |
464 | ||
465 | repeated Goal goals = 3; | |
466 | message Goal { | |
467 | optional GoalType type = 1; | |
468 | optional double target_amount = 2; | |
469 | optional RewardType reward_type = 3; | |
470 | optional string reward_sub_type = 4; | |
471 | optional double reward_amount = 5; | |
472 | optional double target_soul_eggs = 6; | |
473 | } | |
474 | ||
475 | repeated GoalSet goal_sets = 16; | |
476 | message GoalSet { | |
477 | repeated Goal goals = 1; | |
478 | } | |
479 | ||
480 | optional bool coop_allowed = 4; | |
481 | optional uint32 max_coop_size = 5; | |
482 | optional uint32 max_boosts = 12; | |
483 | optional double minutes_per_token = 15 [default = 60]; | |
484 | optional double expiration_time = 6; | |
485 | optional double length_seconds = 7; | |
486 | optional double max_soul_eggs = 13; | |
487 | optional uint32 min_client_version = 14; | |
488 | optional bool debug = 11; | |
489 | } | |
490 | ``` | |
491 | ||
492 | We will need to do a bit of reading. Fortunately, the game has a community wiki, lets look into how contracts should work. I took an | |
493 | [older revision of Contracts wiki page from 2021](https://egg-inc.fandom.com/wiki/Contracts?oldid=13015) and did some slight research. | |
494 | ||
495 | From what I gather, at one point, there was only one set of contract rewards, shared between everyone - then they created a system where | |
496 | beginners will get easier contract goals while more advanced players get harder contract goals. | |
497 | ||
498 | We can put two-and-two together here and infer that `repeated Goal goals` is the legacy contract system - where everyone was on equal footing | |
499 | and `repeated GoalSet goal_sets` is the *new* goal system that is split into Standard and Elite. | |
500 | ||
1e75f355 | 501 | We also learn that in future game versions, they completely reworked how contracts work *yet* again into a grading "bracket" system. Fortunately, |
2a5bb5a5 JM |
502 | we do not have to worry about that in our current target revision. |
503 | ||
504 | Now to get the ball rolling, there is conveniently a starting point set ahead for us already. The developer of game intended to ease new players into | |
505 | contracts by creating a simple & easy contract called [Your First Contract](https://egg-inc.fandom.com/wiki/Contracts/Your_First_Contract?oldid=13547). | |
506 | ||
507 | This page tells us all the information we need to compose our first Contract, so lets try to make one. | |
508 | ||
509 | ``` | |
510 | elif subpath == "get_periodicals": | |
511 | # We don't actually need the information client sends us, | |
512 | # we aren't verifying any stats about client in our server. | |
513 | CurrentPeriodicals = EIProto.PeriodicalsResponse() | |
514 | # [...] | |
515 | Contract = CurrentPeriodicals.contracts.contracts.add() | |
516 | Contract.identifier = "first-contract" | |
517 | Contract.name = "Your First Contract" | |
518 | Contract.description = "We heard you are open to contract work! Help fill this order from the local pharmacy!" | |
519 | Contract.egg = EIProto.Egg.MEDICAL | |
520 | Contract.coop_allowed = False | |
521 | Contract.minutes_per_token = 5 | |
522 | # Lets set expiry time to always be 3 days into future | |
523 | Contract.expiration_time = time.time() + (3600.0 * 72.0) | |
524 | Contract.length_seconds = 3600.0 * 4.0 | |
525 | # The wiki mentions that you cannot get this contract after you reach 5000 Soul Eggs | |
526 | Contract.max_soul_eggs = 5000.0 | |
527 | # We should have the basic metadata set now, lets create the goalsets. | |
528 | FirstSet = Contract.goal_sets.add() | |
529 | Goal = FirstSet.goals.add() | |
530 | # There is only one type of goal in this verison | |
531 | Goal.type = EIProto.GoalType.EGGS_LAID | |
532 | Goal.target_amount = 100000.0 | |
533 | Goal.reward_type = EIProto.RewardType.GOLD | |
534 | Goal.reward_amount = 192 | |
535 | Goal = FirstSet.goals.add() | |
536 | Goal.type = EIProto.GoalType.EGGS_LAID | |
537 | Goal.target_amount = 500000000.0 | |
538 | Goal.reward_type = EIProto.RewardType.PIGGY_FILL | |
539 | Goal.reward_amount = 10000 | |
540 | # Lets now add the Elite table, we can pretty much copy-paste the above here. | |
541 | SecondSet = Contract.goal_sets.add() | |
542 | Goal = SecondSet.goals.add() | |
543 | Goal.type = EIProto.GoalType.EGGS_LAID | |
544 | Goal.target_amount = 100000.0 | |
545 | Goal.reward_type = EIProto.RewardType.GOLD | |
546 | Goal.reward_amount = 500 | |
547 | Goal = SecondSet.goals.add() | |
548 | Goal.type = EIProto.GoalType.EGGS_LAID | |
549 | Goal.target_amount = 500000000.0 | |
550 | Goal.reward_type = EIProto.RewardType.PIGGY_FILL | |
551 | Goal.reward_amount = 10000 | |
552 | return base64.b64encode(CurrentPeriodicals.SerializeToString()) | |
553 | ``` | |
554 | ||
1e75f355 | 555 | Lets try that out in-game now - after waiting for a minute, we see our contract pop up, but I immediately noticed one thing amiss. |
2a5bb5a5 JM |
556 | The contract goals are swapped! I am getting Elite contract rewards for a Standard contract. |
557 | ||
558 | This piece of information now tells us that the first entry in GoalSets refers to Elite rewards and the second entry in GoalSets to Standard rewards. | |
559 | After swapping the sets around, we now see a contract with the corrected rewards. | |
560 | ||
561 | I playtested it a bit and the contract worked as expected. | |
562 | ||
563 | Now, the above code could be a lot neater. For your homework, if you're not skipping to the public source release in the end, you should try to create | |
564 | a contract database and try scheduling them like the game originally did - a "Leggacy" contract every Friday and regular contracts showing up every 1-2 weeks | |
565 | for roughly 2 weeks. | |
566 | ||
567 | ## Conclusion so far | |
568 | We have created a (rather ugly looking) server emulator for the game. It functions, but it needs a lot of work still before we can call it ready. | |
569 | If you have followed this far, give yourself pat on the back - if you actually tried to run this code, give yourself an extra pat on the back. | |
570 | ||
571 | Before I give you the public source to the project, you might want to try your hand at creating a few more things. | |
572 | - "Cloud" save, present a Backup to any new device that just started playing. | |
573 | - Contracts Database and scheduler | |
574 | - Server Event scheduler | |
575 | ||
576 | I apologize if my method of documenting this has been messy, but that's also part of the chaos of reverse engineering, you are constantly learning new things | |
577 | about the project you are currently doing - refactoring becomes an essential part once you have documented the protocol to a comfortable degree. | |
578 | ||
579 | I won't give any promises for a part 2 any time soon, but I will be trying to make this feature complete, so without further ado, here are the git repository links: | |
580 | [github.com](https://github.com/cernodile/reEgg), [git.based.quest](https://git.based.quest/?p=reEgg.git;a=tree;h=refs/heads/master;hb=refs/heads/master). | |
581 | ||
72731d2b JM |
582 | Next time we will dive into apps that use SSL/TLS and making onboarding for your friends easier. |
583 | ||
2a5bb5a5 JM |
584 | Thank you for reading and making it all the way to the end, |
585 | - Cernodile |