After each event we do I have a process now whereby I journal my own observations before seeking feedback from participants. I take all these voicenotes, WhatsApp messages, in-person anecdotes and emails and compile it into hand-written notes before distilling it all into takeaways here.
Normally I’ll do a chronological account of our events and recap them but given how much feedback was generated from this past pair of events (9 pages) I’m going to focus on that instead this time.
What worked
First off let’s focus on the positives:
-
We had a flawless attendance rate at the Ideathon for the first time in history. That’s 35/35 attendees who completed the onboarding steps and then showed up at the event. I’m super happy with this result and attribute it to the screening process and outreach that happened for this event. We fell short of our Spring 2023 Lisbon event (34 vs. 44) but the fact we hit 100% attendance is incredible.
-
The pitches and judging component at the Ideathon was hands-down the best it’s been so far. I deliberately selected active SaaS founders to be judges this time vs. investors from last time and I worked with them beforehand to set the expectation that we wanted primarily constructive feedback to help the teams find and fix the weaknesses in their proposed ventures. We also switched away from a valuation-based judging process to a deliberation-based consensus amongst Judges. Previously the winner was chosen based on the team that achieved the highest project valuation via trades that were proposed and accepted. This time it was purely a consensus vote amongst the judges and that worked way better. Nicole (one of the judges) came up with the name “Dolphin Tank” for the panel because it was like a friendlier version of Shark Tank. This was a massive improvement and I intend to keep this change in place going forward.
-
The lack of a projector turned out to be a blessing in disguise and forced teams to focus on the content of their talk rather than developing a pitch deck. This led to better pitches with more engagement becuase there wasn’t the crutch of the slides to read from.
-
Absent a sponsor to buy lunch we had Zomato Portugal give a 1-yr Pro membership on its app to every participant. While the activation and ordering process had some hiccups the value there was a super nice perk to offer people.
-
I had my buddy Omar play a surprise live musical set at the viewpoint just 5min away from Casa do Impacto when we ended the event. This was a nice way to cap off the evening and I think people really appreciated the music and vibe. Here’s a good video of Omar performing.
-
Being able to delegate the check-in responsibility to my friend Brian was immensely helpful. I made it so an organizer of an event can designate anyone else in the system as an admin with ability to check others in.
-
The Casa do Impacto space was fantastic for the event and loved by everyone - sunny, open, lots of different nooks and breakout rooms as well as an outdoor terrace for working.
In all, lots of positives to celebrate. Now let’s look at where the opportunities to improve exist:
What flopped
-
We got a late start and I took a more laissez faire policy of not being such a stickler holding everyone to the timed milestones. That ended up being a mistake. Multiple people (myself included) believe that having that structure gives us clear landmarks towards which to steer and keeps teams focused. The day lacked structure compounded by the fact that we had issues getting Zomato memberships activated for people so in some cases lunch didn’t arrive until 3pm. This was all entirely on me and I’ll own this for next time with holding us to the published time schedule.
-
We need to front-load the team formation to occur in advance of the event. In spite of the onboarding and asking folks to research problems and projects ahead of time, we still end up spending far too much of the day just figuring out what we’re doing. I’m going to make it so next event we form the teams in advance and at minimum get clarity on the problem (if not the solution) in advance of the event so we can spend the day more on validation and customer interviews.
-
My gambit to try and familiarize participants with the trading platform by using it to conduct audience choice voting proved completely flawed in practice. It was cumbersome and not structurally sound given that teams were of unequal sizes. It was a worthwhile experiment but needs to be rethought at the next one.
-
Only two of the four teams added their projects to the app in advance which further complicated voting. This should in theory become solved by conducting team formation in advance of the event. We had a suggestion to have randomized teams set in advance so they would be of equal size. I’m thinking it actually makes more sense to develop an algorithm that will construct teams of deliberately diverse skill sets with similar cause/problem affinities.
-
This ^^ of course is a delicate balance. Already by asking people to do onboarding steps in advance we whittle down the pool of attendees but IMO this is actually a good thing. I’d rather have 35 people who took the time to do the homework and are motivated enough to want to be there than to have a pool of 60 people none of whom do any prep and show up creating disarray. I believe the solution here involves preordaining teams of 5 deliberately chosen to have a mix of the right skills and affinities for chemistry and allow swaps before the event. We then set a zoom call on Wednesday evening before the event as a meet & greet with breakout rooms for each team. Participants can swap places and trade to a different team assuming both consent. I think this represents the single biggest improvement to make to the format by front-loading problem analysis, team formation and solution ideation to occur in advance of the day.
-
Fundamental housekeeping/orientation steps got lost in the shuffle when the projector failed in the beginning and I went off script. Need to stick to a script even absent the projector and cover some basics. Also, coffee is table stakes at events and that was my bad for failing to verify that it was provided.
-
The food ordering snafu was 50/50. Some people liked the freedom of being able to eat as a team when they pleased. Others liked the structure of all breaking at the same time. In either case the UX of the activations needs to be fixed on the app. I had done a firedrill activation but did not encounter the issues that many people hit.
-
We had a guy somehow get past security having not completed any of the onboarding steps and try to pitch his project. From this I learned that I need to likely conceal the location of the event until the applicant is approved and only reveal to those folks. But I realize I also need to have a clear policy on what constitutes an appropriate project for pitching. This guy had an existing startup and was seeking essentially free labor from our participants which is a hard no. This, once again, should resolve by virtue of front-loading the team formation and having all project leaders submit via the site the Wed before the event via zoom.
-
Our feedback volunteers instead of rotating teams this time stayed for the full hour with the team they started with. On one hand this gave them an in-depth chat with that one person but I think it would be more useful to insist on rotating every ten minutes and impose that time constraint so they get 6x the number of vantage points.
-
The role of the mentors needs to change from naysaying and shooting holes in ideas to become more Socratic method-like in helping to guide teams to question the weaknesses. This too should be less of an issue with front-loading the team formation to Wed but on the morning of Day 0 the idea is going to be decidedly half-baked at best. I’m questioning whether the role of mentor at the Ideathon even makes sense. It might be more helpful to have stations of subject matter experts that are fixed so that teams can (on their own) visit skill-specific stations to get answers they need. ← that probably makes more sense.
-
This is a silly one but I printed 20 large-format posters as well as 20 8.5x11" flyers in advance of the event with QR codes that went directly to the signup page of the Ideathon. But once that was over all those flyers advertising the pair of events were essentially worthless since the page they linked to said “Registration closed.” So I printed new QR codes and went around pasting the on posters to update the link. The real solution though would have been to link to a redirect that could be changed after the fact to automatically point to the subsequent event signup after the first event was over. Lesson learned.
-
On the subject of having better structure and guardrails on the event, I developed The Problemattic Workbook - a printed 11pg set of exercises that we’ll use at future events to step people through the ideation and validation process. This should put things on rails and also develop a meeting of the minds with the team and capture the essence of that in a useful artifact which can be uploaded and referenced from the project page.
-
We never did the Steve Blank “get out of the building” step at this one to have interaction with potential customers in the field which was a shame because we were in a very central area with access to plenty of people to talk with. Time constraints prohibited this but again, we should win back some of the day by front-loading aspects to that Wed evening before.
-
Discord use was almost non-existent leading up to the event. Again this should in theory resolve if we have teams formed, a good orientation and sell people on the value of communicating with their team via Discord in advance of the event. The question has arisen whether using centralized infrastructure like Discord, Passbolt and Trello even makes sense or whether each team should just use whatever tools it wants. Jury is out on this. I see pro’s/con’s to each but I lean towards having at least a minimum set of consistent tools for the sake of continuity and preserving assets and comms in the midst of churn in teams.
-
Need to make it 100% clear to participants of the Buildathon that they need to bring a device to use in getting work done.
Open Questions
OK so here are some open questions for which I would love input:
-
How do we turn this into a true movement that runs under its own propulsion? These events involve an incredible amount of work and expense to throw which is why I currently only host them once per quarter. The work to promote an event is by far the biggest inhibitor of doing more of these. I’ve developed the “Referrals” tab in the admin to enable anyone to easily blast it to their social networks, track who they originate and even earn stake in projects from helping promote. I also implemented a feature this time that only confirms the event once a minimum threshold of approved attendees is achieved and encourages newly-approved applicants to promote it to help make it happen. What other ideas are there to get attendees to promote awareness of the events?
-
Of the people who attended either event, what was your motivation? Did the event(s) meet your expectations? If not, what could be done to make it fulfill its promise?
-
Are the Bounties at all a motivator or are these completely a red herring to you? I’m entirely biased in that I constructed the game mechanics of this whole thing and put a fair amount of time into creating the explainer videos that should demystify everything so I think they make sense but ultimately it only matters how they’re perceived and used. Is the monetary aspect completely irrelevant or is it a strong motivator for you? Either way, why?
I’ll conclude by sharing this testimonial reel we recorded at the event. As much work as it is to build the platform and throw the events it’s these responses that make it all worth it. Thank you to everyone who took the time to record a thoughtful bit of feedback here. If you were at the event or have attended previous events and want to chime in on anything above, I welcome all constructive feedback. Obrigado.
