VestaCP critical flaw leads to hacked servers (Patch already available)
#1
Many users of the VestaCP web stack and control panel have complained that their servers have been hacked one after another and then were abused. See following thread for reference: https://forum.vestacp.com/viewtopic.php?...088ee86e39 or https://forum.vestacp.com/viewtopic.php?...58&p=68543

The VestaCP team has recommended to shutdown the Vesta service to prevent further machines from getting hacked while they investigated the issue and worked on a fix.

The flaw has been identified and fixed already. So there is a patch available. It was a password check function that allowed attackers to gain access to the servers. The VestaCP team has updated all password functions in the patch to make them more secure.

https://forum.vestacp.com/viewtopic.php?...260#p68893 Wrote:The fix has been released just now!
As usually there are 3 ways to update your server:

1. Via web interface
- Login as admin
- Go to updates tab
- Click un update button under vesta package

2. Via package manager
- SSH as root to your server
- yum update / apt-get update && apt-get upgrade

3. Via GitHub
- SSH as root
- Install git / yum install git /apt-get install git
- Then run following commands
Code:
cd $(mktemp -d)
git clone git://github.com/serghey-rodin/vesta.git
/bin/cp -rf vesta/* /usr/local/vesta/

Some information about this indecent. We still don't have working exploit for previous version. But we know for sure that the vector of attack was through a potentially unsecure password check method. Therefore we have completely rewrite password auth function. It's bullet proof now!

Please upgrade your servers as soon as possible.


If you are using VestaCP: please upgrade your server ASAP!

Just saw this in the LET headlines in the morning and thought I post about it here as I know that people here also use VestaCP. @deanhills might this be something interesting for you?
#2
Thanks for the quick update.

You can also use this command to update from command line
v-update-sys-vesta-all

https://forum.vestacp.com/viewtopic.php?f=25&t=16575

I'm also using VestaCP. Hope no one hacked my server. I have to make sure.
Android Fan,Developer and Supporter.
My Websites
http://www.cybapps.com

Thanks to HostDare for VPS
#3
Thanks so much for alerting us @Monad.  I'm obviously a big user of VestaCP.  As far as I know my VestaCP (most VestaCPs by default are or so I thought) is on automatic updates, and it looks OK, but I can't know for sure.  It says it is updated but doesn't provide the date - however looks as though there has been a reboot of my Web server that is a day old - this could have been automatic when the automatic update was applied? Everything in the control panel looks OK and there is nothing that looks out of kilter in the Server report. Do you think I should stop the VestaCP or wait and see first?  I'm almost hesitant to use SSH wondering whether that may trigger something that is not not a problem that has been discovered yet.

What I'll probably do first is to backup all of my WordPress sites through the WordPress interface first, and then figure out by the end of that what my next step should be.  I doubt it will be easy for someone to get access to my WordPress sites, but who knows, not sure what kind of control the hackers get when they hack into VestaCP.

I checked the VestaCP forum discussion thread link that you provided and there are 35 pages of discussion in only 2 days - I'm still busy reading through all of them - your thread is therefore great as it at least gives a framework for the problem and makes some sense out of a tower of Babel of feedback.  The number of affected servers are mind blowing.  And scary.  If one reads a number of those who were affected were using keyless ssh and their ports had been secured.  It's difficult to make sense out of it because there are so many participants in the thread, but I get the feeling that this has to do with recent installations of VestaCP, like during the last 10 days - do you think I am right? If I'm right then it probably would not be a good idea to load a fresh installation of VestaCP until things have been completely sorted?

What worries me too is that I've read a few of the first pages and last pages of that thread and I haven't seen posts by the senior admin who usually feature in threads like that yet.  I hope they will pop up in the other pages.

I've just picked up my first threads with @Imperio. The Admin I know are on top of it. That makes me feel a bit better.  This is reading like a Kafka thriller - wow!  Nervous
#4
@deanhills

You're welcome. When I wrote this topic I decided that I should ping you because I know that you're a big fan and user of VestaCP.

As far as I understood, from the reply I quoted, the VestaCP developer team has already released a fix for the issue. A update is required to install it. So if you have automatic updates enabled and VestaCP says there is no new update available you are good to go (I guess). I never really used VestaCP though and so I have no knowledge of the update process.

The way the servers get abused, it looks like the XOR trojan was installed on them. This is a Linux trojan that abuses servers to attack other servers and computers and try to take them over (SSH bruteforce, etc...). This trojan usually generates processes with totally random names that keep changing all the time (like in the last lines of the PS output from this post). I would suggest to login into your server and make sure it acts normally and that such weird processes aren't running.

If everything is fine you are probably good to go.
#5
Thanks @Monad.  Looks as though my post happened at the same time yours came.  Smile

I learned a lot today about VestaCP security and while I was learning copied some of the discussion below that I thought was noteworthy.  I'm still not completely confident that we're out of the woods yet.  I don't think I'll  do new installations of VestaCP until I am confident the installation script is secure.

Spoiler: show
Just for the record found the following content in the VestaCP Forum thread about the exploit that is noteworthy:

skid-VestaTeam Wrote:Here is what we know so far:
1. The first wave happened on April 4. Servers were infected with /etc/cron.hourly/gcc.sh
2. It was an automated hack
3. CentOS, Debian, Ubuntu all distros are affected it's platform independent
4. We didn't find any traces in vesta and system logs yet
5. On April 7 infected servers started to DDoS remote hosts using /usr/lib/libudev.so.

[...]

What we are doing:
Few users provided us with root access to their servers. We are investigating what happened. We also launched a couple honeypots in order to get full picture of the hack.
https://forum.vestacp.com/viewtopic.php?p=68594#p68594


Imperio-VestCP Team Wrote:Who want provide access to hacked server?
Please, send access via info@vestacp.com


lukapaunovic - Irritated Hacked Member Response Wrote:@imperio

I do not know how u don't realize that no provider will allow hacked server running. i barely convinced ovh to get it up for 10 mins so I can backup data

imperio - VestaCP Team - 8 April Wrote:I think we found a vulnerability. Fix will be today
Source: https://forum.vestacp.com/viewtopic.php?p=68752#p68752

imperio - VestaCP Team - 8 April Wrote: 
Quote:The developers said that they already found the vulnerability
We cant confirm that problem with vesta api, but we will update the password checking


lukapaunovic -Hacked User - 8April Wrote:Its on GitHub

https://github.com/serghey-rodin/vesta/ ... e359cda7dd

It will be on main servers soon
To update now from GitHub:

Code:
cd $(mktemp -d)
git clone git://github.com/serghey-rodin/vesta.git
yes | /usr/bin/cp -rf vesta/* /usr/local/vesta
service vesta restart

install GIT before this

Immediate response by Imperio

Imperio - VestaCP Team 5 minutes after above post Wrote:Need some tests
Source: https://forum.vestacp.com/viewtopic.php?p=68885#p68885

Following which the fix in @Monad's OP was posted:
https://forum.vestacp.com/viewtopic.php?p=68893#p68893


"skid - VestaCP Team - 8 April Wrote:The fix has been released just now!
As usually there are 3 ways to update your server:

1. Via web interface
- Login as admin
- Go to updates tab
- Click un update button under vesta package

2. Via package manager
- SSH as root to your server
- yum update / apt-get update && apt-get upgrade

3. Via GitHub
- SSH as root
- Install git / yum install git /apt-get install git
- Then run following commands

Code: Select all

Code:
cd $(mktemp -d)
git clone git://github.com/serghey-rodin/vesta.git
/bin/cp -rf vesta/* /usr/local/vesta/

Some information about this indecent. We still don't have working exploit for previous version. But we know for sure that the vector of attack was through a potentially unsecure password check method. Therefore we have completely rewrite password auth function. It's bullet proof now!

Please upgrade your servers as soon as possible.

"Imperio - VestaCP Team - 8 Apr Wrote:All virus procesess should be killed and files with virus should be deleted
https://superuser.com/questions/877896/ ... 24#1004724

VestaCP User Question about how the hacking happened - 9 Apr Wrote:  
Quote:How did you got hacked if the port was closed? With the port closed, there is not access to the Web UI.

   If that is true, the only way iam seeing it, is that Vesta repositories were hacked and people installed an exploited version of Vesta.

   When did you installed your VestaCP?

yes thats how the hack is working. it is installed hidden and leaves no logs on the server. (via rep)
i have rkhunter, chkrootkit, clamav, iptables, fail2ban and aide.
none of them reacted so it was installed internally and got by every of the security mechanism.
i installed vesta about 10 days ago on this brand new fresh server.
its ssh secure by pubkey, no root login allowed
vesta webui forced to listen to my ip only (tested and working)
parent id of virus was 1 (systemd)

ALSO i get email on ssh logins. no mails were sent during this time.

and i guess thats why their rep is down now and you cant update currently
Source: https://forum.vestacp.com/viewtopic.php?p=69046#p69046
#6
For those who are following the VestaCP security breach, I found a better quality discussion at LowEndTalk:

https://www.lowendtalk.com/discussion/14...esolved/p1

VestaCP Admin says the patch will solve everything and there is no problem with the script, however some of the systems admin participating in the discussion at LowEndTalk disagree. Their point of view is that the admin at VestaCP haven't found the source of the exploit yet - they're guessing - they can only claim they've fixed the problem if they could replicate what the hackers did. So caution is recommended. The systems admin participating in the discussion at LowEndTalk suggest that everyone disable their VestaCP panels, but I'm hesitating on this. I'm worried that if this infection is dormant that if I log into my VestaCP through SSH in root, that I could trigger it. I'm not convinced yet either that I have a problem. If I find information that compels a shut down of VestaCP, then for me it's much better to start fresh with reloading the OS from the VPS host's control panel and start from scratch without VestaCP. For now I'm not going to do anything. I may be taking a chance, but thought to wait until there is more information.
#7
Ahh! Thanks for this news just updated my vesta cp. Well I already st it for every 2 week but doing now .




Users browsing this thread: 1 Guest(s)