Transcript
Kelly:
Hi, this is Kelly. And I thought I would give a brief demonstration of recall. That's the feature from Microsoft that takes pictures of your screen and then allows you to search for information. It's been talked about in the press and was delayed until recently. Friday. November 22nd, in fact, when it was released as a test version for Microsoft Insiders running the. Development branch. I'm not going to focus on the privacy and security aspects the press has covered that and frankly, I don't know enough about everything. I was just curious how it works from an accessibility perspective and gave it a try. I'm running this on a Lenovo. Yoga slim seven with a Qualcomm Snapdragon processor. Right now you definitely need that sort of processor, so basically. One of the arm based Microsoft Copilot PC's. The blog on Windows Insider has more details about how you turn this on and all the specifications. As I mentioned earlier, recall was released just on Friday, November 22nd. And when I've tried it so far NVDA and Narrator are reading things. JAWS and recall seemed to cause the recall to crash right now. As I said, this is just released so I'm sure the people with Vispero will have JAWS working soon and the blog post releasing. In announcing recall indicated there were some accessibility issues, so this is still a work in progress for everybody, I'm sure. The first time that you launched, recall you will have to go through a brief opt in experience where you're turning it on, indicating that the computer has permission to take screenshots and more. It takes about 20 or 30 seconds. The most important thing to note is just that you'll want to be signing into the computer. Using Windows hello, that's when you use. Your fingerprint. Password face ID or one of the other hello methods. Because each time that you open or do something with recall, you are prompted to sign in with Windows Hello. I've already gone through the setup and you can then launch recall either from an app that's on your start menu called Recall or.
Narrator:
Show hidden icons button, taskbar pane.
Kelly:
As I like to do go into the system tray.
Narrator:
OneDrive OneDrive, recall, preview, recall saving snapshots button.
Kelly:
So here's the system tray entry for recall and we can hear that it's saving snapshots. I can press enter.
Narrator:
Snapshots are being saved on your PC snapshots with potential.
Kelly:
And we get a text about snapshots are being saved.
Narrator:
Open recall button.
Kelly:
If I hit tab once I go to open recall.
Narrator:
Pause until tomorrow button.
Kelly:
Tab again and I get to option to turn recall temporarily off.
Narrator:
More recall in snapshot settings button footer group.
Kelly:
More settings.
Narrator:
Back button snapshots are open recall button.
Kelly:
I'm going to open recall by pressing enter.
Narrator:
No item start, capture pin Edit Windows security window, window security, close button, making sure it's you sign in with Windows Hello to user call and.
Kelly:
So there again, as I mentioned, we are prompted to sign in.
Narrator:
Options Menu button, Recall Preview, win, search edit type your search history. Press delete, remove suggestion.
Kelly:
Little bit of. A little bit of extra chatter and I'm not sure why we get the message about deleting a suggestion since I haven't typed anything yet. But you end up in a search box. You can also then tab around.
Narrator:
Refresh timeline button.
Kelly:
So here's a button for refreshing the timeline. The timeline is basically your images that have been captured.
Narrator:
November 24th, 2024 5:48 AM two minutes session Button 5 memories in session one of one sessions group.
Kelly:
So sessions are basically the as far as I can tell, like. Kind of the time you you've been.
Narrator:
Today, November 24th, 2024, 5:48 AM, two minutes session Button 5 memories. Passion.
Kelly:
Using your computer and the memories are like the snapshots that.
Narrator:
Are in it today. November 24th.
Kelly:
I can hit tab again.
Narrator:
Now button scroll to current time.
Kelly:
This would jump me to the most current time.
Narrator:
Reef it's technology.
Kelly:
And now I'm in my list of snapshots.
Narrator:
One of 229 selected.
Kelly:
One thing I've noticed for some reason is that. Many of these titles for the snapshots don't read automatically the first time you focus them, like there's one that was going to talk about e-mail message that I had.
Narrator:
It's technology.
Kelly:
Open.
Narrator:
200 and 27200 and 22125 of 200 and.
Kelly:
I'm just arrowing back through these and for some reason.
Narrator:
2223 bit stick.
Kelly:
I've found that if I hit home and then a couple of times.
Narrator:
Settings 1 of 229 selected.
Kelly:
So if you noticed the first time I went to the first one, it didn't read anything. Now it does read settings. If I go back to. The end.
Narrator:
Read bits technology, the new Microsoft for call preview Windows Insiders message re bits technology bits technology the new Microsoft.
Kelly:
Now, many of these are reading.
Narrator:
Technology Inbox Kelly at Idea play full view fidelity. Google Chrome 2 Full view Fidelity Google Chrome 223 of 229 selected 222 of 229 selected.
Kelly:
Here again, one's not breathing.
Narrator:
200 two 122.
Kelly:
But you can arrow up and down this list.
Narrator:
Refits technology, the new Microsoft recalled.
Kelly:
I hit end again to go to the last item. From this point if I tab.
Narrator:
Highlighted items group, REITs technology, the new Microsoft recall preview, Windows Insiders message, HTML Group press enter to access highlighted items, press escape to go back.
Kelly:
So this says if I press enter I'll access the highlighted items from this window. I'm just going to tab around once before we. Come back to this outlook.
Narrator:
Classic button commands group.
Kelly:
So for each snapshot, then you kind of have a commands group. And I found that if I use right and left or I can switch between these. The first one said Outlook Classic that is where that snapshot came from and that would open. Outlook.
Narrator:
Click the do button off. Click Click to do 2 of five.
Kelly:
This Click to do I haven't quite figured out how this feature is supposed to work or what it does.
Narrator:
To do 205.
Kelly:
And honestly, turning it on or off so far as far as I can tell, it hasn't made a difference, I'm sure. There's. Because this is supposed to like let you take different actions. As I've read the blog post and such, but I I haven't figured out how this works.
Narrator:
Copy snapshot Button 3 of. 5.
Kelly:
You can also copy the snapshot I hit right arrow to move through these commands delete.
Narrator:
Button four or five.
Kelly:
Delete the snapshot.
Narrator:
Options button 505.
Kelly:
Or options and press enter.
Narrator:
Edit with snipping tool menu item one of one pop up window.
Kelly:
You can you could for example edit the snapshot.
Narrator:
Options button options menu button.
Kelly:
And then here's an options menu. I hit tab for all of copilot.
Narrator:
Snapshots are being saved on your PC snapshots with potentially sensitive info aren't saved unless you choose until tomorrow button menu item.
Kelly:
So this is some of that same things that we had when we were in the system tray.
Narrator:
Pause until tomorrow button.
Kelly:
About pause until tomorrow menu item. There's a nameless item which.
Narrator:
Feedback button.
Kelly:
Appears to be a duplicate of the send feedback item.
Narrator:
Settings menu item snapshots are being saved on your PC. Snapshot options menu, search, Edit, refresh timeline button November 24th, 2024 5:48 AM Now button scroll to current time Redis technology the new Microsoft.
Kelly:
We're going to hit escape. And now we're back to the last entry. Where I was talking about that e-mail message, I have moved back in my timeline to something I was trying yesterday.
Narrator:
Document 1 document one word 25 of 229 selected.
Kelly:
So far I've just been kind of tabbing around the. Recall experience. When I'm new to an application, I like to do that and see what I can tab to move around. I will also try different screen reading commands to see if there's other items I might want to read or explore. That doesn't mean you can't reach them from the keyboard. It means I haven't necessarily figured out how you do that yet, and so I like to use a combination of both screen reading and keyboard commands. When I say screen reading, obviously you do that with the keyboard, and when I say keyboard commands, I mean commands that would be used whether I was using a screen reader or not. In this case I'm running narrator right now, so I'm going to hit narrator and right arrow for the next item off of this word entry.
Narrator:
Image document One word group.
Kelly:
So here is the image that was captured. And. A title has been assigned to it. If I want to get more details, narrators does have a command to try and get an image description. It is not as robust as you might be used to with other solutions yet, but I can hit caps. Control D.
Narrator:
Getting Image caption A screenshot of a computer image text. While you have documented word searches, design, layout, reference mailings, post view, help comments. Editing for share 12 AP ABC ABCD AB ABC SPAR heading 1 heading 2 editor Caplow. Paint aboard Purse Graff Direto off and devices Lenovo.
Kelly:
Now that's a lot of gibberish. Honestly, it's trying to do OCR on the image of the computer screen so. Some of what was read was elements of the word UI, and then I had some Bluetooth settings open them. At the time I was doing this, so reading the screenshot in this way is probably not that useful yet. I have found that if I copy these screenshots and run it through one of the other. Solutions, whether that's jaws, image descriptions, Googles Gemini. Be my eyes or something else you get like you would with any screenshot taking a pretty good description of what's going on. I'm then going to hit Capslock right arrow again.
Narrator:
Highlighted items group. Press enter to access highlighted items, press escape to go back.
Kelly:
So this is highlighted items. I'm just exploring it now so I'm gonna hit caps lock right. Arrow again.
Narrator:
Item one of one Group 1 of 75. We one of one autosave item, one of one Group 2 of 70. 5. Contains one items. Press enter to access items in Group, press escape to. Go back to navigate through available words, press control plus left or right arrow to select multiple words. Press control plus Shift plus left or right arrow.
Kelly:
So now we are getting all of the various texts that was found in the picture. Again, much of this text right here is just elements from the window, like the auto save button right there. And as far as I can tell, the way that narrative presents this is. Whatever copilot has decided to group, sometimes it's one item, sometimes it's a bunch will be called the group, and then there may be items inside. Of there.
Narrator:
Autosave one of one off item, one of one Group 3 of 75. Off one of one item, one of one group I one of one document, one word item, one of one group 5 of 75 contains one items, press enter to document one layout item. One of you have one.
Kelly:
I am going to skip forward. Again. Of one using narrator next item commands for right now.
Narrator:
Press spacebar comments, comments when edit 12, one of one contains one items. Press enter to access.
Kelly:
So what's going to happen here, even though?
Narrator:
Image press spacebar for actions one of one. Kelly Ford item one of one group 51 of 75, contains one items, Kelly Ford, one of one. Bluetooth and devices item 1 of 25 Group 52 of 75 contains 25 items. Press Bluetooth and Devices 1 of 25.
Kelly:
So you'll notice we.
Narrator:
Press space for actions.
Kelly:
You'll notice we went from word to Bluetooth and again, that's because I had Bluetooth settings open as well, and my word window was probably not fully maximize. And so really this is just all of the elements that were in that screenshot, not overly interesting in this case because there's not a lot of meaningful content. But as I said, I like to kind of explore what's around me.
Narrator:
Highlighted items group. Press enter to.
Kelly:
I'm going back to just keyboard navigation and hit enter.
Narrator:
One of one Group 1 of 75. Contains one items. Press enter to access items in Group, press escape to go back autosave item. One of one group 2 of. 70. 5 contains one items. Press enter to access items in Group.
Kelly:
Now I'm using. Left and right arrow to move. Between what again are called the groups are basically the elements that have been broken out from the picture.
Narrator:
Off I don't want one. I don't want to document one word item, one of one group 5 of 75 contains one items, press enter to access items in Group, press it.
Kelly:
If I hit enter.
Narrator:
Document one word, one of one. Press space buffer actions.
Kelly:
So I have a couple of choices at this point I can hit. Control shift right arrow.
Narrator:
Document once selected.
Kelly:
And I've now copied that text again. This is just the document title so not overly helpful in this case, but if this was actually part of my document probably would be more helpful.
Narrator:
Copy menu item, one of three control open with menu item.
Kelly:
I also have choices to copy open with or search the web with this text.
Narrator:
Search the web menu item. Document One word 1.
Kelly:
And then it's all just variation. Variation on the theme.
Narrator:
Press spacebar for action. Document one word item, one of one group, five of seven. Ohh, search item one of.
Kelly:
Again, right and left arrow.
Narrator:
I don't, I don't want. Ad X file I don't want have one home. I don't insert find. I don't want a BABCABAB.
Kelly:
To move through all of these elements, a word window is going to have a lot of UI because of the ribbon and everything else.
Narrator:
Contains one items press.
Kelly:
It's worth repeating that recall came out just Friday, and so far I've kind of just. Moved through the UI, not in a way that would really be practical for using to find information. I'll talk a little bit about the search experience, but I'm still trying to figure out how to use this effectively with the screen reader and read information. As I mentioned, when you open recall.
Narrator:
Search edit type here to search history, press delete to remove suggestion.
Kelly:
You are placed in a search box. Now I took. The text of. My recording so far and created a Word document with that along with some other things that I've been doing on this computer. So we'll try a couple of searches. I'm going to type in the word screen reader.
Narrator:
Screen.
Kelly:
And hit enter.
Narrator:
5 results search complete.
Kelly:
So I heard there were 5 results and my search was complete. If I start hitting tab.
Narrator:
View timeline button. All button on one of four filters group.
Kelly:
Now this filters group would let me I guess limit results to certain types. It's not obvious but. You can use left and right arrow to move between these.
Narrator:
Outlook classic. All button Outlook media player button Google Chrome button off 404 media Player, Outlook classic all buttons on one of four.
Kelly:
And space to toggle. I'm going to leave it with all. If I hit tab again.
Narrator:
File Explorer results. Text matches button.
Kelly:
And now we hear our tab again, says text matches.
Narrator:
Read it's technology, the new Microsoft.
Kelly:
So there is an e-mail message that probably mentioned screen reader.
Narrator:
Recall available accessibility.
Kelly:
I hit right arrow to move through other search results.
Narrator:
Document one word button today at 238 PM 3 or 4.
Kelly:
So this document, one word is a search result again. Because I had taken a bunch of the text from my recording and put it into a Word document to ensure I had a search result. Here's where it really gets interesting. And again, I haven't either figured out a good strategy of what to do, so I'm going to hit enter on this. The challenge I face is I'm not really sure what's going to be a great way to figure out what about that document. Was referenced or why I came up in my search results?
Narrator:
200 document one word 200.
Kelly:
From my previous explorations, I know if I hit tab once.
Narrator:
Highlighted items group document one word group. Press enter to access highlighted items, press escape to go back.
Kelly:
I'm on this entry that says press enter to go to highlighted items.
Narrator:
We title bar item, one of seven Group 1 of 54 contains 7 items. Press enter to access items in Group, press escape to go back to navigate through available words, press control plus left or right arrow to select multiple words. Press control plus.
Kelly:
Shift plus left or right arrow. Now it's kind of that same. Mishmash of all the word UI.
Narrator:
Share I don't want 11454 find I don't want.
Kelly:
And everything else.
Narrator:
Heading item, one of one group.
Kelly:
That is really not.
Narrator:
Replace item one of one group, paste item one of one group 26.
Kelly:
Interesting to move through to figure out.
Narrator:
Contains one items press.
Kelly:
What was here? So I have no big finish because I'm still trying to figure out a meaningful way to go from. A result in my timeline, especially for searching to reading the actual meaningful text from the search result.