I’ve been a Siri power user for a long time, mainly because I’ve always liked using voice as an interface. But, legacy voice assistants like Siri, Google Assistant, and Alexa were never well integrated enough or reliable enough to actually save time. Maybe 1 in 5 commands end up executing as smoothly as you expected, but the most useful thing they do is play a song or set an alarm. The advent of LLMs seemed like a great opportunity to push the state of the art forward a notch or two!
Our goal is to do 2 things better:
1) Deeper integrations with productivity-related apps you use every day, like calendar, email, messages, whatsApp, and soon Google Docs, Slack, and phone calls.
2) Better memory of each user based on their past conversations and integrations, so Martin can start to anticipate parameters in the user’s commands (e.g. text the guy from yesterday about the plans we made this morning)
A great way that our early users use Martin is having morning syncs and evening debriefs with the software. At the start/end of each day, they’ll have a 5-10 minute sync about their TODOs for the next day, and Martin will brief them on upcoming tasks and news they’re typically interested in.
Something else Martin does which is unlike other voice assistants is it can have full text conversations with your contacts on your behalf from its own phone number. For example, you can tell it to plan a lunch with a friend, and it can text back and forth with that friend to figure out a time and place. After the text conversation between your friend and Martin is over, Martin reports back to you via a notification and a text. You can also monitor all of its messages with your contacts in the app.
We started building Martin exactly 1 year ago, during our YC batch. It’s definitely a hard product to “complete" because of the many unsolved technical challenges, but we’re making progress step by step. First was the voice interface, which Siri still hasn’t gotten right after more than a decade. We have 2 modes: push-to-talk and handsfree. Handsfree is great for obvious reasons. We’ve gotten our latency down to only a couple seconds max for most commands, and we’ve tuned our own voice activity detection model to minimize the chance of Martin cutting you off (a common problem with voiceGPTs). But, even then, Martin may still cut you off if you pause for 3-5 seconds in the middle of a thought, so we made a push-to-talk mode. For those cases where you want to describe something in detail or just brain-dump to Martin, you might need 20-30 seconds to finish speaking. So just hold down, speak, and release when you’re done—like a walkie talkie.
We’ve also had to tackle a very long tail of integrations, and we want to do each one well. For example, when we launched Google calendar, we wanted to make sure you could add a Google Meet link, invite your contacts to the events, and access secondary calendars. And, you should be able to say things like “set reminders leading up to the event” or “text Eric the details of this event.” So, we pretty much release one new major integration every month.
Finally, there’s the problem of personalization / LLM memory, which is still very unsolved. From each conversation that a user has with their Martin, we try to infer what the user is busy with, worried about, or looking forward to, so in their next “morning sync” or “evening debrief”, Martin can proactively suggest to-dos or goals/topics to discuss with the user. Right now, we use a few different LLMs and many chain-of-thought steps to extract clues from each conversation and have Martin “reflect” periodically to build its memory. But, with all that said we still have a lot of work to do here, and this is just a start!
You can try Martin by going to our website (https://www.trymartin.com) and starting a 7 day free trial. Once you start your trial, you’ll get an access code emailed to you along with the download link for our iOS app. After you enter your access code into the app, you can integrate your calendar, contacts, etc. If you find Martin useful after the trial, we charge our early users (who are generally productivity gurus and prosumers with multiple AI subscriptions) a $30/month subscription.
We can’t wait to hear your thoughts. Any cool experiences with Siri, things you wish a voice assistant could do, or ideas about LLM memory, tool calling, etc. - I’d love to discuss any of these topics with you!
We recently got our CASA Tier-2 compliance done (Cloud Application Security Assessment). We've also gone through Google's OAuth compliance process for every new integration we add that's related to Google. These assessments scan our app and make sure that our software meets pretty stringent standards when it comes to data security and encryption, and that we're not using the data for anything other than the specific features we promise (i.e. not sharing or selling to advertisers, etc.). You can read more about CASA here (https://appdefensealliance.dev/casa). We haven't gone through SOC2 yet, but planning on soon once we have a few more integrations.
Does personal information get sent to OpenAI or Claude as part of the functionality? Can users request deletion of their data, and if so, what is the process? Are there specific protocols in place to ensure security? (i.e. Do you use encryption at rest?).
Deleted Comment
As you develop your messaging, I wanted to share the questions I had as I think a lot of users will ask the same:
1. What powers Martin? Is it a custom LLM or powered by OpenAI, Anthropic? 2. Is any of my data ever used in training? 3. Will I always be notified before new texts / calls / actions are taken on my behalf? Does the AI present as me or are my contacts aware that it's an AI assistant that may provide incorrect information? 4. Can I easily and quickly remove all my data and context?
I’ve been putting it through its paces and it’s handling some complicated requests correctly the first time. For example:
“There is an art festival in my city this weekend. They have a jazz stage my wife and I would like to check out. Find the schedule for each day and create one event every day it’s happening. In the event description put the schedule for each day, and invite my wife.”
It got it right the first time. Pretty amazing.
I see some folks saying it’s just a “wrapper for an LLM” like that’s easy to do. LLMs are not faerie powder that just work for every use case. The personal assistant use case is extremely difficult, which is why the big players haven’t done it yet.
So bravo for the bravado and actually making it work. Privacy is a concern, but honestly I’m not that worried that you can find out which art festival I’ll be at this weekend. But an oncology appointment? I might.
You should create a system where you cannot access user data, and it can never be shared with third parties. Make that system open source to prove it. Give up the potential upside of using this data for revenue so that Martin becomes what it can be. Otherwise, I’ll never feel confident telling Martin anything I don’t want advertisers to know.
We can certainly publish more privacy guarantees in the future - thanks for the suggestions. Our business model is subscriptions, so we won't be going anywhere near ads or data sharing.
https://ios.gadgethacks.com/how-to/60-ios-features-apple-sto...
Apple’s strength and weakness is that there is only one Apple way to do things for ease of use and for it to “just work”. However, not everyone fits into their cookie cutter design regardless of how good it is. Customization and options beyond what Apple does tends to be the way to go.
they pivoted mid yc it sounds like. yc chooses founders not ideas.
I'm sure it won't be long before we see apps that listen, record any "Hey Siri" they hear, and then synthesize that voice to give your phone commands to "tell me my passwords", or more insidious and difficult-to-detect commands.
It seems Apple's new version will be facing this problem too.
Deleted Comment
- How did you solve the long-term memory problem? What kind of issues are you facing with scaling the number of tools?
- I like the idea but there's one crucial thing missing for me. I will happily pay for your app if it lets me bring my own API keys/ endpoints for models that I can host, so that I know my data is private and secure.
- Right now, we use a combination of RAG and chain of thought for storing memories. At different time intervals, we'll create memories at different levels of granularity. For example, at the end of every conversation, we'll embed some vectors based on specific commands from the user. At the end of each day, we'll have the LLM reflect on key questions related to a user's routine. And every few days, it'll reflect on the user's short/long term goals. This has worked to some degree, but we're still in the very early stages of figuring out how to do long-term memory for an assistant.
- Scaling the number of tools is definitely a struggle since we want to make our integrations as thorough as we can. It takes time, so we just try to keep growing the list consistently. We have an internal goal of adding at least one new major integration every month.
- Love the idea of bringing your own API keys/endpoints. We've gotten this feedback before, so we'll seriously consider it in our next few sprints!
As someone who is very interested in using this, may I make two suggestions:
1. Have a list of integrations somewhere on the homepage. It might be there, but if so I missed it. I immediately wanted to know if it can integrate with Obsidian, for example, or Omnifocus. I'm sure others will want to know if "email" means Google only, or Outlook, etc.
2. Make the trial longer. When I see 7 days, what I immediately think is "not enough time to really test this". I'm a busy person, I'm not going to change habits overnight, and unless this thing will immediately integrate into my daily routine (it won't), I'll probably only use it casually the first few times. It would be much better to give me more time to test it. (This is not business advice - maybe I'm wrong and 7 days is better to actually convert users! I'm just giving my immediate reaction.)
My main problem with startups around this is that it’s just a big ask to get access to all my data and store it in their cloud.
...after almost 10 years of internal stagnation and killing Siri's other upgrade projects: https://www.macrumors.com/2023/04/27/report-details-turmoil-...
It's just baffling. Same goes for Microsoft and how utterly unusable Cortana is, these features should be more than afterthought integrations that limp along because they're too cheap to get rid of. The slow "evolution" of voice assistants is pretty much ensuring that nobody wants to use them, at least among the people I know that own smartphones. Something tells me that AI won't be the selling point Apple thinks it is, especially when anyone with a web browser can use ChatGPT for free.
Not sure how it will affect this startup.
It has been a few years and my Android assistant is still dumb. I would have expected the iOS/Android assistants to be much better by now.
I am guessing its against their philosophy to release product that only works sometimes from the get go. Thats why they have been so demure about whole AI stuff.
But have you used Siri?
“Only works sometimes”, yes if all you ever ask is “what time is it”, what’s the weather, and maybe 5-6 other canned prompts.
Half the time it doesn’t even understand or catch what’s being said. (Mainly talking about the HomePod, it generally works fine on an iPhone)
Siri is a great example of Apple being perfectly fine letting a product languish so what might’ve been great 10 years ago is a punchline now.