GuideAnts: Open-Source AI OS

Speaker 0: Got Doug Ware. Anand, guys, I'm gonna put a little bit of a timer, for us because I wanna make sure we get everybody in. So just look for me, and I'll give you guys an update on where you are, inside of your presentation. But we got Doug Ware with Guide HINTS OSS. Speaker 1: Thank you. Thank you. Man, I feel like I'm on Jeopardy Anand the clock is talk. Node power. Okay. Speaker 1: Alright. Andrew I'm working on 1 monitor. It looks good. So, I read the charter of this group, Andrew was real clear that this was supposed to 2026, like, demo oriented. So I have slides, but LLM not Jonah show any of them except for as part of the demo. Speaker 1: My name is Doug Ware, and I have a company called Elumenotion Anand, been in Atlanta for a long timeouts anybody here ever seen me before? That's awesome. I rate span the dot net user group for over a decade, and I organized a lot of community in hospitals in, like, a whole different space. This is a different community for 544K, and, I think that's that's really cool. So my goal for you today is to, clone this repo and contribute to GitGyan or rate the very React, make the star count go up. Speaker 1: If I get home tonight and it's Skills 2026, then I mill cry. OSS, you can make my day right now. The way I found out about this event was through this website, everyeventever.com, and this website is actually maintained by the platform that I'm gonna show you guys today. So it Ware, like, kind of a cool Generation thing. I got an mill. Speaker 1: It told me about this. I was like, hey. I'd like SPEAKER. So here I am here today. And this is, if if you go to this website, you'll be like 1 of the dozens of people a month that stumble by there. Speaker 1: So as I introduce myself, so, Illumination is a company that I started in 2007 Anand I ran as a consulting teaching and software business up until 2019, and we did, Office '3 65 and SharePoint and a lot of solutions around that. I was a Microsoft MVP, for that. I've actually been an MVP for 4 different decades for different things, most recently for AI serverless. But I left the Microsoft community because I was dissatisfied with what they were building. I used to use their stuff, but, I'm not into Copilot or any of that stuff hosting because I just don't like City. Speaker 1: And we Code talk about why that is, but this product that I'm gonna show you today, this open source project, was sort of born out of my desire to have a thing that work the way that I wanted it to. So, we're just Jonah jump right in. I Store off by making the bold claim that AI, it's a pun, guides plus assistance 544K Foreman AI. So Guidance Notebooks is an operating system. So the 1st thing is it's built around files and has excellent support for files and folders in place 544K organizer. Speaker 1: People use AI, use Node, AI, you know. So OSS of really good file viewers for different kinds of file formats Claude the ability to, edit and view Office Elumenotion the fly and work together. I was really this is an Runs 2 different open source projects, 1 called Euro Offering, another 1 called Jonah Office, and I was super excited to have an open source project that I could integrate with, to be able to do this. When you bring your documents in to be able to work with them, you need to prepare them for, AI to be able to use Pham. And so silently in the background, we extract all the content and turn it into markdown and index it and Diagram and all that fun state, vectorizing City. Speaker 1: And that is done with an open source project called Logging. Anand if we look at our PDF right here and look at the extracted text, use can see Node looks really nice. This is a hard problem. This is 1 of the things that it does. And so Week, it 1st of talk, it was built around AI when I'm using IDENTIFICATION, I'm probably doing work Anand my work is around documents whether those are code files or presentations or budgeting spreadsheets or whatever it happens to be that I'm working with. Speaker 1: I use a lot of markdown too. Anand then the next thing we need to be able to do in an operating system is have Diagram, right, that can run against our content. So in AI Notebooks, we have the concept of guides And a guide is like a skill mla an MCP server plus sandbox files plus knowledge AI, and it's intended to completely encapsulate an AI application in a way that I can use it in different places for different things Find different applications like that memory Event, every site, which has just got a job that runs and it's doing agent stuff Pham maintain the AI. You don't chat with it, but there's a AI Pham maintains that. So an example of what a guide looks like is this PowerPoint 1. Speaker 1: This PowerPoint guide is divided up into this top Live, agent Anand this is the system prompt Pham describes the guide. And then it has a crew of these other System, which are also guides in and of themselves. And these crew members are reusable, and I use them all over the place. So we put a lot of time and effort into search and media creator and diagrams because they're, you know, things that we use, a lot of the place a lot of places. And you can just see from this user interface that I could spend 10 minutes talking about the stuff that's in the AI builder. Speaker 1: But in order to have a program work well, it needs to be able to connect to lots of teaching. And so tool sources that we can consume include web APIs, client actions, which I'll show you in a 2nd. Those are things that happen in the browser, sandbox modules, MCP connections, local functions, or you can just put in a raw scheme in there. And this handles lots of different auth flavors, and a lot of work went into making sure that this work is isolated between each program so that 1 author key isn't leaked to another files that are used by Jun notebook. Anand a guide lives inside of a container called a notebook along with HINTS group. Speaker 1: So when we go to use 1 of these things, we create a new notebook and we get 1 of these things. So I'm gonna go into this Jun, Guidance Presentation, which is based on the PowerPoint AI here. I'm not gonna spend the whole talk talking about this prevent, but I wanna set the stage for the next topic. So here at the top of this, you can see we have these buttons for the different AI sorts of things that you could Doug, chat, image generation, image to image, audio processing, ASR, embeddings, which you don't see up here. And there's a setting system that'll let us route these 2026 OpenRouter, Microsoft Foundry, Hugging Face Inference, Local AI, Anthropic, Anand another 1 that I'm forgetting. Speaker 1: Anand you can mix and match those. So this demo that I'm running right here to now is running on this laptop and is not doing Lovable. Ware I use this in my home machine, I'm using QN and nothing but local models but that machine's got 2 GPUs Anand 96 gig of RAM. And 1 of the goals of this was to build a platform that we could take and give to somebody with a potato powered laptop and, you know, divvy it up, give it somebody with a big power computer Store decompose into a big Kubernetes cluster of lots and lots of different containers Anand actually scale. So before I show you, I'm gonna talk about the architecture a little bit. Speaker 1: This is the Console talk for the application that I'm running right now Jun it's all running on my laptop. There's 1 open-source. Normally I've opened up the SQL Server because I was doing some development. But in here are some of these containers that I was talking about. And a lot of these are based on really amazing open source inject. Speaker 1: Your office for the implement serverless, Coding, which is an IBM Air. Docling is 1 of the greatest projects in the work. Teaching Anand Playwright in order to read websites and do searches without an API key. The AI image within the Python sandbox and other AI of runtime sandbox. The user interface, explains Use for diagrams Anand then a database server. Speaker 1: And like I said, this is all running on 1 machine built you can run this. In fact, we do in the Claude, run it in container apps Anand Kubernetes clusters Anand the pieces can move and you can decompose them in lots of ways. And Store importantly, you can add new sandboxes into them because each 1 of these is working on the files that I showed use, AI? File sit at the middle of it. And so, when the AI sandbox is working on some files, the point Mill can be working on the same files, but there are different processes Anand they have process boundaries around them, which lets us AI incorporating TypeScript execution agent AI new types of sandboxes really easily. Speaker 1: So PlantUML is a Java app of all logging, right? It's Java API like. It's got a Jun proxy sitting in front of it Find our Stripe Execution Agent and it just works. I don't know how I'm doing on time. Okay, so let's look at this chat. Speaker 1: So this chat was done with Speaker 2: AI Speaker 1: GLM 5.2 which just came out. I ran on Open-source and I gave it a 1 shot prompt teaching it to use this project Folder, which is my code rate Claude from GitHub Anand just create a PowerPoint deck from it for me. This is power AI-Powered API guide. I could address any of these System individually, but the top level orchestrator can use any of them and delegate the workout. So I just gave it that prompt and it took 14 steps and it ran for a very long time Anand it used CodeExecutor a lot and it used diagrams a lot and Date used slideshows and then use CodeExecutor Anand it made a slide deck with 18 slides in it and a whole bunch of diagrams. Speaker 1: And this is what the slide deck looks like in the PowerPoint viewer built it's easier to look at in the HTML version of it that it built. This image here came out of the Repos. It was on 1 of the Create files it picked as did this 1. This 1 is an image from rate AI test that it selected for that. This is a plant UML diagram and GLM built this entire deck Find all these images and curated and then did all that in 1 shot Anand it took about 20 minutes to operate. Speaker 1: Node, did it do a good job? The output is correct built we also need Drift we're gonna have an operating system mla way to manage that. So Jun of the things that you can do with AI is see their actual usage and be able to tune them. So if we look at the reporting for Power AI here and look at that conversation, we can see it used about 1000000 tokens Anand we can see all the different tool calls that it did Anand how much they cost Anand we can drill into it. And if we look at our presentation that we built here, we can see it actually did 4 24 individual tool Skills. Speaker 1: Anand so if we were looking to optimize this application, we might go, why did it feel like it needed to read web pages all these AI? It was in some sort of loop. I don't have to wonder though because I can drill into the invocation details and see what happened since we're running out of time Anand they're gonna whistle me off here. Like I said, you can publish this as MCP servers as guides. Use can service them in other applications AI this demo Here's. Speaker 1: I was gonna start Runs. Oh, here it is. This is a AI that's running our SaaS application embedded into a game. Start a new game, eat and grow, live as long as you can. And that ASR took place in Azure power our talk end Andrew it's using the Ware commander guide, which is sending commands to the web browser, which is then controlling the game. Speaker 1: And this is fun. I'll just let this guy sit and run all day AI to see what a high score can get, built more state of practical version of what you can use this Foreman we if you look at my website, there's a Demos here of the Power BI navigator, and this is a guide that understands how to use this report workspace that can answer questions and navigate around. And we use it Event here in this Jun. I can do things like Speaker 2: show me the usage report. Speaker 1: This guide is brand new. I'm still working Node, and it's mostly Graph. But it's it's in here to demonstrate it. Soon, it'll be able to actually start new conversations and and do stuff. And the other thing is AI I'm using, like I said, I'm using GLM against Stripe router OSS Date can AI of be slow. Speaker 1: Last thing I wanna show, I'm not gonna wait for that to finish, built I just think is fully, really neat. Power here, this is author. Week have the MCP server set up for that same diagrams guide Anand I Air Cursor said, hey, make a diagram of a Console talk Anand City use that MCP server and drew this. And then the last thing that I'll show you just to try to stick the landing is when we publish 1 of these things, another thing that we can do is proxy OpenAI compatible hosting through it. And we can do this with the respect API Anand with Anthropic too. Speaker 1: So you could set up a guide with your coding rules in it and point Claude code at Anand cloud Code, all of its traffic Doug go through your guide and you would track and be able to see literally everything that went on and inject other concerns into it and add tools or do that with any other any other features. So, yeah, thanks for having me. Please Mishra that star button, clone the repo. If you'd like a tour of the app, using me on LinkedIn. I'll help you get set up. Speaker 1: What's coming, that our intern is what power intern Patterns is working on. I'm gonna get a pull request Prisma better 1 click installer that'll ask you about your hardware and help you configure it Anand, we need testers because supporting Jun Rockum, NVIDIA CUDA, Avoilan, Apple Matter, I don't have all that kind of equipment. I can't test it all myself. So we build it and then look for somebody to talk food it and get you tickets. Us. Speaker 1: So, thank you. Sure. Speaker 3: Yes. Speaker 1: They just upload them or you span, I didn't show it in the Demos, but you can link to a mount on a network share? So the demo I did with the Power presentation, I linked it to my repo Anand that's how I got all the files. Speaker 3: So somebody but somebody needs to curate the content, like, Speaker 2: to make the process? Yeah. Speaker 1: I mean, it it it depends on what you're doing and what the purpose of the guide is. So, like, we have guides for doing SDLC processes, and what they do is they walk product managers through how to run the teaching, and then they transcribe it and build a document. So in that case, the only implement that's going into them, you know, in normal like, they're uploading a meeting, and then they're producing a bunch of files that come off the other side. Yes, sir. Speaker 3: For the PowerPoint generation, 1 of the biggest I Node. Speaker 1: Rate? It Weeks, doesn't it? AI. No. That demo and that all of those guides are in the Repos. Speaker 1: So when you spin it up, you'll get all those guides and see how they're all built. But the PowerPoint 1 has got 3 Employee that I put in it. You can make more through the AI. You can edit them. There's also 1 in there for Word that demonstrates a different way of doing it, which I just don't have time to talk about, but that 1 works off of a template as well. Speaker 1: So, if I've got something AI need my letterhead on, I take the implement, AI run it through that assistant, and it just styles it for me using, you know, using Python, basically. Built, at the end of it, I've got because, you know, of the storage Anand the organization, I've got it in the project Anand the notebook, and there's file generate and timeouts OSS other other fun stuff too. But when you start looking at the downside of things like skills, which just describe what you wanna do with maybe some other files, you start to realize why being able to actually also provide executable code and services and tools that go along with that package Store is really an important thing because, if your agent is figuring out how to do the formatting for your builders, but it's supposed to be the same every single time. Speaker 2: Right? You're just burning money. And the best thing that'll happen is it's right. The the worst thing that'll happen is Senta% of the time that is wrong, and you gotta spend an hour tweaking it, which, you know, at which point the users go, this rules crap. Right? Speaker 0: I'm not Speaker 2: using Copilot. Speaker 1: Alright. Thank you Doug.