All ShareFile users started the week with an unexpected password reset.
On December 1, Citrix made the decision to reset (without proper warning) the passwords of every ShareFile user, including customers using multi-factor authentication mechanisms.
As reminder, the views, thoughts, and opinions expressed on this website belong solely to the author, and not necessarily to the author’s employer, organization, committee or other group or individual.
On Saturday, Citrix put the following message on status.sharefile.com:
— Nicolas Ignoto (@citrixguru) December 3, 2018
To summarize, Citrix is saying that many users are using credentials that may have been stolen in previous breaches (not related to Citrix) this year, and they decided to reset the passwords of most, if not all, ShareFile users. The message was purely preventive, and no sign of breach into Citrix systems or any suspicious activity was mentioned by the company.
My first thought was that the massive password reset was most likely because Citrix had been breached. It seems like an extreme, knee-jerk reaction to only be related to other third party breaches. Citrix assured me that was not the case, but they haven’t officially confirmed that ShareFile was not hacked. Let’s keep in mind that organizations tend to keep breaches secret to avoid being in the spotlight.
When the password reset was announced, the only public notification was the one published on Sharefile status page. Nothing was published on the Citrix blog, tweeted from @Citrix or @ShareFile, or displayed on the ShareFile login page. That sounds like a good plan when you are doing something impacting hundreds of thousands of users, right? What could go wrong?
We had to wait 48 hours for Citrix to publish an official blog post about the incident.
See below the full message from Stan Black, CISSP & CSIO of Citrix:
— Citrix (@citrix) December 4, 2018
Here is the most important part of the message:
So Citrix updated the root cause on Monday and added that they discovered suspicious activities (credential stuffing) for some of the ShareFile accounts over the weekend. It took Citrix a long time to notify all of its customers. Some ShareFile admins first received an email over the weekend, see blow.
And Citrix slowly sent password reset emails to all ShareFile users over the past few days. As an example, I only received my reset link on Tuesday evening, more than 72 hours after the incident. And like most people I thought that it looked a lot like a phishing scam.
@briankrebs Any info about the @sharefile email that was sent out to all customers? Email looks legit and https://t.co/lzSuJYO166 has same info. Sounds like a breach, but no official word yet. pic.twitter.com/ywMmJlb6J8
— BJ Beier (@bjbeier) December 3, 2018
All the while, many customers voiced their surprise and disappointment via Twitter.
For most, notifications of the incident were not communicated properly. When users logged in on Monday, the sign-in screen simply said that their passwords were incorrect and mentioned nothing about the need for a reset. A lot of users are using password manager software to automatically populate logon information. To their surprise, it was not possible to login without a clear explanation.
Customers using API or SFTP were welcomed with outages on Monday without knowing what was going on. For those who tried to reset their password before they received the email from Citrix, the emails with the link were heavily delayed. Most customers ended up with an influx of calls from their users to the support.
Citrix also decided to implement scheduled password reset. That means that they will decide when and how often to reset the passwords of your users. The only option offered by Citrix to avoid the password reset schedule is to configure your account with multi-factor authentication mechanisms. While I initially thought that this could be a great idea, it turns out scheduled password resets are not a best practice. According to NIST and Brian Krebs, scheduled password resets tend to result in users picking weaker passwords over time to make it easier for them to remember and keep track.
Why would you force users to regularly change their passwords, which goes against current NIST guidance?
— Stephen Repetski (@srepetsk) December 3, 2018
Following this incident, I have suggestions for Citrix and ShareFile teams to consider moving forward:
- Don’t do that again! 🙂 or if you have to. Do it properly!
- Only reset the password for compromised accounts
- Build IP address blacklisting, suspicious activity notifications, and captcha capabilities within your solutions
- Display a warning or a link on the logon page to inform users of global changes (maybe a top-nav banner notification?)
- Boost messaging specs, so that notifications don’t take 72 hours to send
- Be clear, quick, and transparent in your communications and updates. Post an article and tweet immediately about the incident. Don’t wait 48 hours and angry customers to communicate about it.
- Consider nixing regularly scheduled password resets, as they are not recommended by industry experts
- Make sure that all ShareFile administrators are getting your alerts
- Encourage your users to use multi-factor authentication
Hopefully, Citrix will learn from this mishap and take the corrective actions to avoid another incident. I’ve never seen Microsoft or Google doing something like that to that many users and it would definitely cause a massive backlash. I am still not 100% convinced that ShareFile was not hacked. I hope Citrix will provide more details in the next few days about what happened during the past weekend.
Note: Citrix published an article on Monday praising the success of ShareFile but quickly removed. Some users mentioned that it was odd to publish such article a day after a massive mishap.
Stay tuned for more info about this incident. I will update the post if needed.