Today I woke up to find one of the servers had been hacked
It seems on server WEB1 somebody gained access they php (thru an insecure php scrip I am sure) and gained httpd access, which allow them access to php files with a 666 or 766 permissions ( meaning httpd can write to it).
They ran a script that removed “\” (back-slash) from php files code, rendering scripts unusable.
Thankfully we had backups, and due to the additional security measures in place, the hacker (or cracker to be politically correct) was unable to gain further access.
We sill 100% unsure to which script he came in, but to be safe mod-security was added, lets hope it keeps any further attacks at bay :/