tag:status.seesaw.me,2005:/historySeesaw Status - Incident History2024-03-18T19:29:43-07:00Seesawtag:status.seesaw.me,2005:Incident/200809982024-02-26T11:42:09-08:002024-02-26T11:42:09-08:00Occasional crash for some iOS 17 Users<p><small>Feb <var data-var='date'>26</var>, <var data-var='time'>10:19</var> PST</small><br><strong>Investigating</strong> - Some iOS 17+ users (iPad and iPhone) have reported occasional crashes when creating complex drawings and using the eraser in the Creative Canvas. This is due to a change in Apple's iOS operating system, and we are currently working with Apple to resolve this issue.</p>tag:status.seesaw.me,2005:Incident/200438102024-02-22T10:41:52-08:002024-02-22T12:43:25-08:00Messaging: Issue with real-time updates<p><small>Feb <var data-var='date'>22</var>, <var data-var='time'>10:41</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Feb <var data-var='date'>22</var>, <var data-var='time'>09:02</var> PST</small><br><strong>Update</strong> - We've confirmed that the issue has resolved itself. An upstream provider had an outage from 2/21/24 13:57 PT to 2/21/24 19:01 PT. Our logs show normal activity resumed by 19:15 PT, and we've made an improvement to prevent this issue from occurring if the provider were to have an issue again.</p><p><small>Feb <var data-var='date'>22</var>, <var data-var='time'>09:01</var> PST</small><br><strong>Update</strong> - We've confirmed that the issue has resolved itself. An upstream provider had an outage from 2/21/24 13:57 PT to 2/21/24 19:01 PT. Our logs show normal activity resumed by 19:15 PT, and we've made an improvement to prevent this issue from occurring if the provider were to have an issue again.</p><p><small>Feb <var data-var='date'>21</var>, <var data-var='time'>18:32</var> PST</small><br><strong>Investigating</strong> - A small set of users are likely not receiving real-time updates in Messaging. The team is looking into this, and the current workaround is to refresh to receive new messages. Thank you for your patience.</p>tag:status.seesaw.me,2005:Incident/196613902024-01-09T15:00:26-08:002024-01-09T15:00:26-08:00Elevated API Errors<p><small>Jan <var data-var='date'> 9</var>, <var data-var='time'>15:00</var> PST</small><br><strong>Resolved</strong> - All services are operational, and back to healthy state.</p><p><small>Jan <var data-var='date'> 9</var>, <var data-var='time'>14:05</var> PST</small><br><strong>Monitoring</strong> - The new change has been rolled back and we continue to monitor the reliability of our services.</p><p><small>Jan <var data-var='date'> 9</var>, <var data-var='time'>14:00</var> PST</small><br><strong>Update</strong> - We are continuing to work on a fix for this issue.</p><p><small>Jan <var data-var='date'> 9</var>, <var data-var='time'>13:56</var> PST</small><br><strong>Identified</strong> - We're experiencing an elevated level of API errors and are currently looking into the issue. This might impact site load performance.</p>tag:status.seesaw.me,2005:Incident/194071902023-12-13T13:08:25-08:002023-12-13T13:08:38-08:00SMS Notifications : Issue in sending some notifications<p><small>Dec <var data-var='date'>13</var>, <var data-var='time'>13:08</var> PST</small><br><strong>Resolved</strong> - We have confirmed that issue is now resolved and reliabilities are back up to 100% since 2:45pm Pacific, 2023-12-12</p><p><small>Dec <var data-var='date'>12</var>, <var data-var='time'>14:45</var> PST</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Dec <var data-var='date'>11</var>, <var data-var='time'>07:30</var> PST</small><br><strong>Investigating</strong> - We are currently aware of an issue communicating with our SMS Gateway provider, resulting in potentially missed SMS notifications for new posts. <br /><br />The team is investigating the root cause, and expecting a fix by Dec 12, 2023 (noon, pacific time)</p>tag:status.seesaw.me,2005:Incident/191399962023-11-16T08:15:00-08:002023-11-16T10:12:33-08:00Issues loading Marketing Website<p><small>Nov <var data-var='date'>16</var>, <var data-var='time'>08:15</var> PST</small><br><strong>Resolved</strong> - Seesaw's marketing website (web.seeaw.me, seesaw.com) experienced loading issues from 8:13AM PST - 9:30AM PST.</p>tag:status.seesaw.me,2005:Incident/191303582023-11-15T11:30:00-08:002023-11-15T11:51:18-08:00Site Loading Issues Between 11:30-11:45am PT<p><small>Nov <var data-var='date'>15</var>, <var data-var='time'>11:30</var> PST</small><br><strong>Resolved</strong> - Some users may have experienced issues with the site/app loading (500 errors) between 11:30-11:45am PT. The issue is now resolved.</p>tag:status.seesaw.me,2005:Incident/191212132023-11-14T13:57:05-08:002023-11-14T13:57:05-08:00Issues Loading Standards & Curriculum Alignments Search Tool<p><small>Nov <var data-var='date'>14</var>, <var data-var='time'>13:57</var> PST</small><br><strong>Resolved</strong> - The issue is resolved and the Standards & Curriculum Alignments Tool should be loading successfully now.</p><p><small>Nov <var data-var='date'>14</var>, <var data-var='time'>12:23</var> PST</small><br><strong>Monitoring</strong> - We are seeing improvement with the Standards & Curriculum Alignments Tool site loading issue. In some cases, refreshing or reloading the page will resolve the issue.</p><p><small>Nov <var data-var='date'>14</var>, <var data-var='time'>12:22</var> PST</small><br><strong>Update</strong> - We are continuing to work on a fix for the Standards & Curriculum Alignments Tool site loading issue. In some cases, refreshing or reloading the page will resolve the issue.</p><p><small>Nov <var data-var='date'>14</var>, <var data-var='time'>11:45</var> PST</small><br><strong>Identified</strong> - Some customers are experiencing loading issues with the Standards & Curriculum Alignments search tool. In some cases, refreshing or reloading the page can work. We are resolving the issue with our third-party web provider and will provide updates as soon as we have more information.</p>tag:status.seesaw.me,2005:Incident/189684472023-10-30T10:30:05-07:002023-10-30T10:30:05-07:00Email and SMS notifications delayed<p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>10:30</var> PDT</small><br><strong>Resolved</strong> - All historical backup of notifications have been cleared and delivered. We consider this incident to be resolved, and continue to monitor the reliability of our services.</p><p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>10:28</var> PDT</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>10:16</var> PDT</small><br><strong>Monitoring</strong> - A fix is in place and all the notifications from earlier today should have been delivered at this time. We are continuing to monitor the stability of our fix and reliability.</p><p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>08:45</var> PDT</small><br><strong>Identified</strong> - We have identified the root cause of the issue. New notifications are no longer delayed, and we are actively working on clearing out the backup in the older notifications</p><p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>07:20</var> PDT</small><br><strong>Investigating</strong> - We are currently experiencing delays in outbound email and SMS notifications for users, since 1:30pm UTC / 930am Eastern / 630am Pacific. Our team is currently investigating the root cause. More updates will be posted shortly.</p>tag:status.seesaw.me,2005:Incident/185382692023-09-18T21:18:14-07:002023-09-18T21:18:14-07:00[Investigation] Site slowdowns and connections issues<p><small>Sep <var data-var='date'>18</var>, <var data-var='time'>21:18</var> PDT</small><br><strong>Resolved</strong> - AWS has stated the majority of its networking latencies and errors have been resolved. Our error monitoring also corroborates their updates. We are resolving this issue but will continue to monitor as well.</p><p><small>Sep <var data-var='date'>18</var>, <var data-var='time'>16:33</var> PDT</small><br><strong>Monitoring</strong> - AWS has reported that their actively working on resolving networking latencies and errors. They anticipate this to be fully resolved in the next 60-90 minutes. We will continue to monitor as well.</p><p><small>Sep <var data-var='date'>18</var>, <var data-var='time'>12:35</var> PDT</small><br><strong>Investigating</strong> - We are investigating site degradation (slowness and connection issues) that have been impacting some users on the site since ~11am Pacific time. This is potentially tied to a degradation in network performance in our regional data center in US-WEST2 that AWS is actively investigating and fixing forward.</p>tag:status.seesaw.me,2005:Incident/181737542023-08-17T07:24:30-07:002023-08-17T07:24:30-07:00Investigating Site Slowness and Connection Issues<p><small>Aug <var data-var='date'>17</var>, <var data-var='time'>07:24</var> PDT</small><br><strong>Resolved</strong> - Site connection / slowness issue resolved. There has not been any known customer impact since ~7am PT today.</p><p><small>Aug <var data-var='date'>17</var>, <var data-var='time'>07:07</var> PDT</small><br><strong>Monitoring</strong> - The slowness/connection issues are resolved and we are investigating the root cause of the site degradation that impacted some users between ~5am PT to ~7am PT.</p><p><small>Aug <var data-var='date'>17</var>, <var data-var='time'>06:54</var> PDT</small><br><strong>Investigating</strong> - We are investigating site degradation (slowness and connection issues) that have been impacting some users on the site since ~5am PT.</p>tag:status.seesaw.me,2005:Incident/181539362023-08-15T09:58:37-07:002023-08-15T09:58:37-07:00Unable to Send Family Invites via SMS<p><small>Aug <var data-var='date'>15</var>, <var data-var='time'>09:58</var> PDT</small><br><strong>Resolved</strong> - Family invites should now send correctly via SMS. If you previously sent an invite via SMS that was not delivered, you can re-send the invite now (via SMS or any method) or remove and re-invite so that the family invite properly sends.</p><p><small>Aug <var data-var='date'>15</var>, <var data-var='time'>09:40</var> PDT</small><br><strong>Monitoring</strong> - A fix for sending family invites via SMS text message has been implemented. Family invites should now send correctly via SMS. We are monitoring to ensure the fix.</p><p><small>Aug <var data-var='date'>15</var>, <var data-var='time'>09:13</var> PDT</small><br><strong>Investigating</strong> - Currently we are unable to send family invites via SMS text message to new families without existing Seesaw accounts. As a workaround, teachers can invite families using email, QR code printouts, or invite link. We also recommend re-sending an invite via one of the other methods if you recently tried to invite families via SMS.</p>tag:status.seesaw.me,2005:Incident/180876822023-08-08T14:44:05-07:002023-08-08T14:44:05-07:00Investigating Site Slowness and Connection Issues<p><small>Aug <var data-var='date'> 8</var>, <var data-var='time'>14:44</var> PDT</small><br><strong>Resolved</strong> - Issue resolved. There has not been any known customer impact since ~11:15am PT today.</p><p><small>Aug <var data-var='date'> 8</var>, <var data-var='time'>12:25</var> PDT</small><br><strong>Update</strong> - We are continuing to investigate the site degradation (slowness and connection issues) that impacted the site between ~10:50am PT - 11:15am PT. There has not been any known customer impact since ~11:15am PT today.</p><p><small>Aug <var data-var='date'> 8</var>, <var data-var='time'>11:33</var> PDT</small><br><strong>Investigating</strong> - We are investigating site degradation (slowness and connection issues) that impacted the site between ~10:50am PT - 11:15am PT. The slowness is temporarily resolved and we are investigating the root cause.</p>tag:status.seesaw.me,2005:Incident/179674792023-07-26T11:00:11-07:002023-07-26T11:00:11-07:00[Investigating] We are currently investigating issues with Google Sign in on the Seesaw Website.<p><small>Jul <var data-var='date'>26</var>, <var data-var='time'>11:00</var> PDT</small><br><strong>Resolved</strong> - We have resolved the issue impacting Google Sign In to the Seesaw website.</p><p><small>Jul <var data-var='date'>26</var>, <var data-var='time'>09:59</var> PDT</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:status.seesaw.me,2005:Incident/177221032023-06-29T11:45:08-07:002023-06-29T11:45:08-07:00[Investigating] We are currently investigating issues with Google Sign in on the Seesaw Website.<p><small>Jun <var data-var='date'>29</var>, <var data-var='time'>11:45</var> PDT</small><br><strong>Resolved</strong> - This incident is resolved.</p><p><small>Jun <var data-var='date'>29</var>, <var data-var='time'>10:45</var> PDT</small><br><strong>Investigating</strong> - We are currently investigating this issue. <br /><br />Users can continue to sign into Seesaw via Google SSO on the mobile applications with no issues.</p>tag:status.seesaw.me,2005:Incident/167535812023-04-04T17:24:12-07:002023-04-04T17:24:17-07:00We are currently investigating issues connecting with Clever<p><small>Apr <var data-var='date'> 4</var>, <var data-var='time'>17:24</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Apr <var data-var='date'> 4</var>, <var data-var='time'>14:27</var> PDT</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:status.seesaw.me,2005:Incident/159334282023-01-26T08:29:50-08:002023-01-26T08:29:50-08:00Intermittent 504 Errors with CSV Rostering<p><small>Jan <var data-var='date'>26</var>, <var data-var='time'>08:29</var> PST</small><br><strong>Resolved</strong> - The issue has been resolved and customers have been reporting that things are now working as expected with CSV Rostering.</p><p><small>Jan <var data-var='date'>25</var>, <var data-var='time'>20:58</var> PST</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results to ensure there are no more CSV Rostering 504 errors coming through.</p><p><small>Jan <var data-var='date'>25</var>, <var data-var='time'>10:38</var> PST</small><br><strong>Identified</strong> - The issue causing the CSV Rostering 504 errors has been identified. A fix is being worked on.</p><p><small>Jan <var data-var='date'>25</var>, <var data-var='time'>08:15</var> PST</small><br><strong>Investigating</strong> - We are currently investigating an issue where some admins get a 504 error when attempting to upload rosters via the CSV rostering method.</p>tag:status.seesaw.me,2005:Incident/158300842023-01-13T08:15:49-08:002023-02-07T16:57:30-08:00Cannot Playback Videos that Were Recorded from Web/Android on iOS devices<p><small>Jan <var data-var='date'>13</var>, <var data-var='time'>08:15</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jan <var data-var='date'>12</var>, <var data-var='time'>13:30</var> PST</small><br><strong>Update</strong> - Videos that are recorded on the Web can now be played back on iOS devices. A fix for video playback for videos on recorded on Android is in review.</p><p><small>Jan <var data-var='date'>12</var>, <var data-var='time'>10:09</var> PST</small><br><strong>Identified</strong> - We identified an issue where users cannot playback videos that were recorded on the web/Android on iOS devices.</p>tag:status.seesaw.me,2005:Incident/113192182022-09-29T17:30:47-07:002022-09-29T17:30:47-07:00Email Sending Delays<p><small>Sep <var data-var='date'>29</var>, <var data-var='time'>17:30</var> PDT</small><br><strong>Resolved</strong> - The issue with our downstream email service provider is resolved and all delayed emails have been sent.</p><p><small>Sep <var data-var='date'>29</var>, <var data-var='time'>16:45</var> PDT</small><br><strong>Update</strong> - The issue with our downstream email service provider is resolved and we are currently sending email notifications. We will update again when all delayed emails have been sent.</p><p><small>Sep <var data-var='date'>29</var>, <var data-var='time'>15:02</var> PDT</small><br><strong>Identified</strong> - We have identified an issue where email notifications are delayed due to an issue with our downstream email service provider. All email notifications will be sent, but users may experience a delay in email delivery. We update again when the delay is resolved.</p>tag:status.seesaw.me,2005:Incident/113022972022-09-28T15:00:26-07:002022-09-28T15:00:26-07:00Potential delay in real-time Messaging updates<p><small>Sep <var data-var='date'>28</var>, <var data-var='time'>15:00</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Sep <var data-var='date'>28</var>, <var data-var='time'>15:00</var> PDT</small><br><strong>Update</strong> - [Resolved] At approximately 1:30PM Pacific, our downstream service provider resolved the issue on their end. Real-time Messaging updates are now fully operational. Please email help@seesaw.me if you're experiencing any issues.</p><p><small>Sep <var data-var='date'>28</var>, <var data-var='time'>10:21</var> PDT</small><br><strong>Investigating</strong> - We are currently investigating issues with real-time Messaging updates as there is an issue with a downstream service provider.</p>tag:status.seesaw.me,2005:Incident/111302982022-09-15T19:30:47-07:002022-09-15T19:30:47-07:00Messages Inappropriate Image Incident<p><small>Sep <var data-var='date'>15</var>, <var data-var='time'>19:30</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved. Read a letter from our CEO/co-founder here: https://web.seesaw.me/blog/credential-stuffing-incident</p><p><small>Sep <var data-var='date'>15</var>, <var data-var='time'>13:39</var> PDT</small><br><strong>Update</strong> - **9/14 Incident Report**<br /><br />*Summary of the issue:*<br />- Late on September 13, isolated Seesaw users were subjected to a coordinated “credential stuffing” attack. These individual compromised accounts were then used to send a message with a link to an inappropriate image.<br />- *Seesaw was not compromised.* The attack was a result of a coordinated attempt to guess individual user account passwords, sometimes known as a 'credential stuffing' attack. In a credential stuffing attack, publicly available compromised emails/passwords that are re-used across services are used to gain access to individual user accounts.<br />- *Seesaw is safe to use.* Within the first few hours of learning about the attack, we took swift action. We completely disabled messaging, blocked the attacker, made sure the image was removed and no longer accessible, and proactively alerted impacted users as well as our wider community. The incident has been resolved. <br />- *Less than 0.5%* of Seesaw users were affected. Seesaw blocked the attack swiftly to prevent the message from being distributed widely. <br />- We have no evidence to suggest the attacker performed additional actions or accessed data in Seesaw beyond logging in and sending a message from compromised accounts. We are conducting a thorough investigation and will share updates if any new information is discovered.<br /><br />*What Happened?*<br />- A message containing a link to an inappropriate image was sent from individual compromised Seesaw user accounts.<br />- *Seesaw was not compromised.* The attack was a result of a coordinated attempt to guess individual user account passwords, sometimes known as a 'credential stuffing' attack. In a credential stuffing attack, publicly available compromised emails/passwords that are re-used across services are used to gain access to individual user accounts.<br />- *Less than 0.5%* of Seesaw users were affected. Seesaw blocked the attack swiftly to prevent the message from being distributed widely.<br />- We have no evidence to suggest that the attacker performed additional actions in compromised Seesaw user accounts beyond logging in and sending a message.<br /><br />* How Seesaw Immediately Responded* <br />- Ensuring the safety and privacy of our teachers, students, and families was our top priority.<br />- Within the first few hours of learning about the attack, we took action. Seesaw’s swift response to this attack included the following steps:<br /> - As we resolved the issue, we completely disabled our messaging feature to ensure no one else saw the inappropriate message. As of September 15, messaging has been fully restored.<br /> - We successfully removed the inappropriate message from all accounts and coordinated with Bit.ly and AWS to ensure that the inappropriate image is no longer accessible.<br /> - We proactively reset the passwords of all accounts we know to have been compromised and notified all impacted users.<br /> - Throughout the day, we released multiple public statements, publicly provided status updates on our website: https://status.seesaw.me/, coordinated with security officials, and notified all customers.<br /> - We adjusted our detection and blocking rules to ensure similar attacks are prevented in the future.<br />- In addition to the above, we have taken a number of mitigation steps to prevent a similar attack in the future through refinements to Seesaw’s rate limiting, alerting, blocking, content detection, and login systems.<br />- We'll be reviewing other steps we can take in the coming days to help users secure their accounts further.<br /><br />**Here’s What This Means for Your Users** <br /><br />*Is Seesaw safe to use?*<br />Seesaw is safe to use. The safety and privacy of our teachers, students, and families is our number one priority and we take it extremely seriously. Seesaw was not compromised, and the incident has been resolved.<br /><br />*Is Seesaw Messages secure and safe to use?*<br />Yes, this incident occurred as a result of unauthorized access to Seesaw accounts. Seesaw, including the messaging feature, was not compromised.<br /><br />*How do I know if my account was compromised?*<br />If your account was compromised, the Seesaw team sent you an email. We proactively reset the passwords of all accounts we know to have been compromised. We have also adjusted our detection and blocking rules to ensure similar attacks are prevented in the future. <br /><br />*Why was messaging turned off?*<br />As soon as we identified this attack was taking place, our first priority was to secure the safety of teachers, students, and families. While we resolved the issue, we disabled the messaging feature to prevent the message from being distributed widely. Before turning messaging back on, we took action to block the attacker’s access and made sure the image was removed and no longer accessible.<br /><br />*How do I ensure that the image has been removed?*<br />We have removed the inappropriate image link from all messages and taken many other actions to ensure it is inaccessible (details can be found here: http://status.seesaw.me). In a few instances, if the message was already loaded in a web browser or one of our apps, it may have been cached on devices. To completely remove the image, users can follow these steps:<br /> - We recommend all users *refresh their web browsers and restart their mobile app*. <br /> - On mobile, we recommend all users update devices to the latest version (version 8.1.2) or re-launch Seesaw by completely closing out and re-opening the Seesaw app. (Here are instructions to close apps for iOS and Android devices). <br /> - iOS: https://support.apple.com/en-us/HT201330<br /> - Android: https://support.google.com/android/answer/9079646?hl=en#zippy=%2Cclose-apps<br /><br />*How can I take extra precaution and reset my password?*<br />Any user can always reset their password at https://app.seesaw.me/#/reset_password. <br /><br />*Where can I get more information or support?*<br />If you have any questions or concerns, please reach out to us here: https://help.seesaw.me/hc/en-us/requests/new. Thank you for your patience while our team worked round the clock to get these additional security measures in place.</p><p><small>Sep <var data-var='date'>15</var>, <var data-var='time'>07:17</var> PDT</small><br><strong>Update</strong> - Messages is ON for all customers. We are continuing to monitor.</p><p><small>Sep <var data-var='date'>15</var>, <var data-var='time'>06:44</var> PDT</small><br><strong>Monitoring</strong> - We are turning Messages back ON for all customers starting at 7 am PT and continuing to monitor.</p><p><small>Sep <var data-var='date'>14</var>, <var data-var='time'>18:37</var> PDT</small><br><strong>Update</strong> - We have removed the inappropriate image link from all messages and taken many other actions to ensure it is inaccessible. However, in a few instances, if the message was already loaded in a web browser or one of our apps, the message may have been cached on your device. To ensure that no one has access to the inappropriate message, we recommend all everyone *refresh their web browsers and refresh their mobile apps*. On mobile, you can update your device to the latest app version (version 8.1.2, released today) and re-launch Seesaw OR close and re-open the Seesaw app.<br />Here are instructions to close apps on mobile devices:<br />- iOS: https://support.apple.com/en-us/HT201330 <br />- Android: https://support.google.com/android/answer/9079646?hl=en#zippy=%2Cclose-apps<br /><br />We are also emailing all affected users with these instructions as well. We plan to re-enable Messages tomorrow after overnight monitoring.</p><p><small>Sep <var data-var='date'>14</var>, <var data-var='time'>15:01</var> PDT</small><br><strong>Update</strong> - *What Happened*<br />- Late on September 13th, Seesaw was subjected to a coordinated “credential stuffing” attack.<br />- Seesaw was not compromised; however, isolated individual user accounts were compromised and used to send an inappropriate message.<br />- Widely available compromised emails/passwords that were reused across services were used to gain unauthorized access to Seesaw accounts.<br />- We have no evidence to suggest this attacker performed additional actions or accessed data in Seesaw beyond logging in and sending a message from these compromised accounts.<br /><br />*What Data May Have Been Accessed*<br />- An isolated number of Seesaw user accounts were compromised in this attack, and only a portion of those were able to send the inappropriate message before the attack was blocked.<br />- We have no evidence that the attacker performed additional actions in Seesaw beyond logging in and sending a message from these compromised accounts.<br /><br />*What We Have Done*<br />- As soon as we identified this attack was taking place, we took action to block the attacker’s access to these accounts.<br />- We completely disabled the messaging feature to ensure no one else saw the inappropriate message as we worked to resolve the issue. <br />- We removed the inappropriate message from accounts where it was sent.<br />- We proactively reset the passwords of all accounts we know to have been compromised, and have notified impacted users already.<br />- We adjusted our detection and blocking rules to ensure similar attacks are prevented in the future.<br />- We have coordinated with Bit.ly to ensure that the link to the inappropriate image is no longer accessible in any email notifications that may have been sent.<br /><br />*What You Should Do*<br />- Maintaining the security of Seesaw accounts is paramount. While we work to proactively prevent these kinds of attacks, you can help by making sure that all user accounts use unique passwords that are not used in other services.<br />- If you are an administrator, you can let your parents and teachers know:<br /> - Seesaw was not compromised; however, isolated individual user accounts were compromised and used to send an inappropriate message.<br /> - This incident was the result of an outside actor; the messages were not created by the sender.<br /> - If your account was compromised, your password was reset and you have already received an email notifying you of this.<br /> - It is essential that you always use best practices to ensure your password is secure: https://www.cisa.gov/uscert/ncas/current-activity/2018/03/27/Creating-and-Managing-Strong-Passwords<br /> - Seesaw takes protecting your security and privacy seriously and we have a number of measures in place to protect the integrity of your information. You can learn more here: https://help.seesaw.me/hc/en-us/articles/203258429-How-Seesaw-keeps-student-data-safe<br /><br />* Next Steps* <br />- We will re-enable messaging when we’ve confirmed that the inappropriate message is no longer accessible from our servers.<br />- We are actively monitoring the situation and will be putting additional mitigation measures in place to prevent this and future attacks of this type.<br />- We will be scanning databases of known compromised passwords and resetting the passwords of users who may have re-used passwords as a proactive additional security measure (and preventing use of these passwords in the future).<br />- Please always use a unique password for accessing Seesaw and any other online account or service. Never reuse an old password or use the same password. Consider using a password manager for added security.<br />- We will continue providing updates here.</p><p><small>Sep <var data-var='date'>14</var>, <var data-var='time'>13:40</var> PDT</small><br><strong>Identified</strong> - We believe there may be some caching cases where the previously sent inappropriate images may still be accessible to some customers. At this time we do not believe there are any new messages with the inappropriate image being sent. We have turned the Messaging feature off while we investigate the issue further. We are continuing to monitor and investigate the situation and will post a more detailed update shortly.</p><p><small>Sep <var data-var='date'>14</var>, <var data-var='time'>10:42</var> PDT</small><br><strong>Monitoring</strong> - We have reset passwords for affected individuals and confirmed that links are no longer accessible. At this point, we have re-enabled the Messages feature. We are continuing to monitor and investigate the situation and will post a more detailed update shortly.</p><p><small>Sep <var data-var='date'>14</var>, <var data-var='time'>10:24</var> PDT</small><br><strong>Update</strong> - As of 9:15am PT: Users who received the link via an email notification will no longer be able to access the link.<br />As of 10:15am PT: We have reset passwords for and sent password reset emails to all affected accounts.</p><p><small>Sep <var data-var='date'>14</var>, <var data-var='time'>08:05</var> PDT</small><br><strong>Update</strong> - It was brought to our attention that a link to an inappropriate image was being shared via the Messages feature. It appears that specific accounts were compromised by an outside actor. <br /><br />We take this incident extremely seriously. We’ve turned OFF Messages temporarily for all users while we investigate, to prevent further spread of this image from being sent or seen by any Seesaw users.<br /><br />- We have removed this inappropriate link from any messages that reference it<br />- We are resetting the passwords of any users who had their account compromised <br />- We have taken measures to prevent additional accounts from being compromised<br />- Any user can always reset their password at: https://app.seesaw.me/#/reset_password<br />- Our team continues to monitor and investigate the situation<br /><br />If you have any questions or concerns, please reach out to us at https://help.seesaw.me/hc/en-us/requests/new. <br /><br />We will continue to provide updates on https://status.seesaw.me/ as we investigate and manage the situation. <br /><br />—The Seesaw Team</p><p><small>Sep <var data-var='date'>14</var>, <var data-var='time'>07:41</var> PDT</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Sep <var data-var='date'>14</var>, <var data-var='time'>06:13</var> PDT</small><br><strong>Investigating</strong> - [Investigation] We have turned off our messaging tool temporarily while we investigate an issue.</p>tag:status.seesaw.me,2005:Incident/109007692022-08-24T08:18:13-07:002022-08-24T08:18:13-07:00Unable to Sign in with Clever and Classlink SSO on the Web<p><small>Aug <var data-var='date'>24</var>, <var data-var='time'>08:18</var> PDT</small><br><strong>Resolved</strong> - This issue has been resolved.</p><p><small>Aug <var data-var='date'>24</var>, <var data-var='time'>08:07</var> PDT</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Aug <var data-var='date'>24</var>, <var data-var='time'>07:24</var> PDT</small><br><strong>Investigating</strong> - We are investigating an issue where Clever and Classlink SSO results in an endless loop on the web version of Seesaw. Clever SSO on iOS is unaffected.</p>tag:status.seesaw.me,2005:Incident/102202062022-06-10T11:19:55-07:002022-06-10T11:19:55-07:00Customer Conversations Not Successfully Migrated to Messages<p><small>Jun <var data-var='date'>10</var>, <var data-var='time'>11:19</var> PDT</small><br><strong>Resolved</strong> - This issue has been fully resolved and all impacted customers should now see their previous conversations in the new messaging experience.</p><p><small>Jun <var data-var='date'>10</var>, <var data-var='time'>10:57</var> PDT</small><br><strong>Update</strong> - We have re-migrated the conversations for all impacted schools and are monitoring to make sure everything is fixed.</p><p><small>Jun <var data-var='date'>10</var>, <var data-var='time'>08:38</var> PDT</small><br><strong>Monitoring</strong> - We've fixed the root cause so we expect this issue should not impact any customers moving forward. We are now re-migrating <br />the conversations for all previously impacted schools.</p><p><small>Jun <var data-var='date'>10</var>, <var data-var='time'>08:02</var> PDT</small><br><strong>Identified</strong> - We’re investigating an issue where some schools are reporting that they are missing their previous conversations after turning on messages. We’ve identified the root cause (a recent infrastructure upgrade) and will be fixing this ASAP, then remigrating conversations for all impacted customers. <br /><br />For customers that experience this, none of their conversation data is missing or lost, and we will be properly migrating previous conversations over as soon as possible. Customers can continue using messages without any impact to previous conversations.</p>tag:status.seesaw.me,2005:Incident/102004372022-06-09T07:24:19-07:002022-06-09T07:24:19-07:00Potential delay in email notifications<p><small>Jun <var data-var='date'> 9</var>, <var data-var='time'>07:24</var> PDT</small><br><strong>Resolved</strong> - This issue has been resolved and email notifications are being delivered normally.</p><p><small>Jun <var data-var='date'> 9</var>, <var data-var='time'>06:55</var> PDT</small><br><strong>Update</strong> - We are seeing signs of recovery from our downstream email provider and are working through the delayed emails. We expect all delayed email notifications to be correctly sent in the next hour or earlier.</p><p><small>Jun <var data-var='date'> 9</var>, <var data-var='time'>05:37</var> PDT</small><br><strong>Identified</strong> - We have identified an issue where some email notifications are delayed due to an issue with our downstream email service provider. All email notifications will be sent, but users may experience a delay in email delivery. <br /><br />You can view the status of our email provider here: https://status.mailgun.com/</p>tag:status.seesaw.me,2005:Incident/101690972022-06-07T10:36:11-07:002022-06-07T10:36:11-07:00Potential delay in email notifications<p><small>Jun <var data-var='date'> 7</var>, <var data-var='time'>10:36</var> PDT</small><br><strong>Resolved</strong> - This issue has been resolved and email notifications are being delivered normally.</p><p><small>Jun <var data-var='date'> 7</var>, <var data-var='time'>09:22</var> PDT</small><br><strong>Identified</strong> - We have identified an issue where some email notifications are delayed due to an issue with our downstream email service provider. All email notifications will be sent, but users may experience a delay in email delivery.</p>tag:status.seesaw.me,2005:Incident/100327792022-05-19T20:09:28-07:002022-05-19T20:09:28-07:00Messaging Notifications Delayed<p><small>May <var data-var='date'>19</var>, <var data-var='time'>20:09</var> PDT</small><br><strong>Resolved</strong> - We’ve processed all backed up notifications and the notifications delay issue should now be fully resolved for all customers.</p><p><small>May <var data-var='date'>19</var>, <var data-var='time'>18:00</var> PDT</small><br><strong>Update</strong> - We’ve increased the speed of the process and now expect the delayed notifications issue to clear up for customers in under 3 hrs (or earlier). <br /><br />We’ll continue to monitor and will update again when the issue is fully resolved for all customers.</p><p><small>May <var data-var='date'>19</var>, <var data-var='time'>16:30</var> PDT</small><br><strong>Update</strong> - We made a few changes to address the issue and are monitoring the improvement. We are working through the backed up notifications which takes some time, so customers may not see improvement immediately. <br /><br />We expect the issue to clear up in the next ~6 hrs (or earlier) for customers. In the meantime, we continue to recommend refreshing to view new messages.</p><p><small>May <var data-var='date'>19</var>, <var data-var='time'>12:00</var> PDT</small><br><strong>Monitoring</strong> - We've been investigating an issue where new messages notifications are delayed and not showing up in real-time.<br /><br />In the meantime, customers will need to refresh their browser/app to view new messages. <br /><br />Note that this issue impacts the new Messaging product only, Legacy Inbox & Announcements is unaffected.</p>