Lindsay Lee
work
About
resume
Beam
me up!
⇥
fleetsmith

Custom Apps for IT Admins

How I helped admins set up employees with custom software they need to be productive at work–remotely and securely.

Screenshot of Fleetsmith Custom App page on a MacBook
About fleesmith 🏢
Fleetsmith is a web app that helps IT Admins automate the lifecycle of their organization's Apple devices from setup, maintenance, and security.
The Challenge ✨
Design a way for IT Admins to deploy apps not available in our Catalog to their employee's work devices.
My Role
I led the design approach and partnered with a PM and researcher to test and iterate solutions, engineers to bring to life, and Sales and Support to rollout to customers.

TL;DR

The ability to remotely install apps we didn't offer in our Catalog of content to devices was one of Fleetsmith's most frequently and longest requested features. Although a known feature in the device management space, my goal was to craft a thoughtful solution tailored to our specific customers' needs, goals, and that they'd love.

It was one of the first projects I worked on at Fleetsmith, cracked open a window into the secret lives and minds of IT Admins, and was overall a fast and fun project to work on.

Results

Over 5 months, I took us through quick ideation, multiple rounds of testing and interviews, and beta releases before we launched the feature to all customers in December 2019.

Why was this important to solve?

Fleetsmith was a series B startup with the goal of owning the device management space. It was loved by customers for its ease of use, robust Catalog of configurable apps and device settings, and focus on security.

However, IT Admins at mid- to large-size companies needed the flexibility to install additional software we didn't provide–known in the space as "custom apps." Delivering a great solution would allow us to better support and retain existing customers at larger companies, while also opening up new business opportunities and conversations upmarket.

The Challenge ✨

How might we help admins deploy custom software in a way that doesn't require a lot of time-consuming or confusing steps?

What problems were we trying to solve?

Getting employees set up with the devices and apps they need at onboarding is a big part of an IT Admin’s job, and they typically have a checklist of things they need to complete to consider a device completely "set up."

In our current world, IT Admins weren't able to automate the deployment of anti-virus and proprietary apps to their org's devices through Fleetsmith. Many resorted to manually installing  apps on every single new employee's device which was time-consuming.

The onboarding workflow

In order to help IT Admins complete their device setup checklists–specifically, their needs around anti-virus and proprietary apps–the product manager and I focused our approach on solving 2 key problems:

Designing for IT Admins

When considering the needs, behaviors, and mental models of IT Administrators–Fleetsmith's core users–one does not simply empathize and know things. So I relied a lot on getting in the (virtual) room with customers via research and learning from the small crew of IT Admins around me.

Among our customer base, a few key distinctions I considered were their level of technical skills and experience in device management.
For this effort, the Product Manager and I aligned on prioritizing creating a solution non-technical or newer admins could easily use, while ensuring we provided functionality technical and experienced admins need. This would allow us to build a solution both accessible and valuable to the broadest audience.

Constraints & considerations

There were a couple factors guiding this effort:
‍

  • Time sensitive
    Our goal was to release support before end of Q4 2019–a little less than 5 months.
    ‍
  • Getting our core experience just right
    Our biggest obstacles were validating our assumptions and delivering a thoughtful solution that satisfied customer expectations.
    ‍
  • Managing key customer relationships
    Some customers had been long awaiting custom apps support. So we worked closely with them during testing and evaluation phases.
    ‍

Defining our approach

The Product Manager, Design Manager, and I kicked off the project with multiple working sessions to align around what business outcome we wanted to prioritize for the first release, define product design requirements, open questions, and align our overall strategy.

To deliver a meaningful feature in 5 months, we decided to rigorously prioritize requirements essential to deploying a custom app to devices, test with customers to make evidence-based decisions, and iterate until we got it right.
Our goal: Enable IT Admins to create and deploy custom apps that require any combination of kernel extensions or privacy preferences.

Testing concepts & assumptions

To kickoff, I set out to understand the mindsets of our IT Admins, learn who they are, and their level of familiarity with technical aspects of custom apps and our current system. So, I quickly mocked up a point-of-view to test with customers, identify gaps, and assess how well our proposal met their needs.

Creating a custom app

The structure
I hypothesized when IT Admins create a custom app, they'd want to be able to quickly see all their options in view, add what they need, or learn more from brief copy and Help links.

I assumed IT Admins were familiar with how Catalog Apps worked in Fleetsmith, so structuring the "Create Custom App" page using similar content hierarchy and patterns would feel consistent, enable them to find information quickly, and make it feel like they're building their own app.

Package URL
The fastest, leanest way customers could deploy custom app was by self-hosting their app and providing us a URL to the package they want to deploy. We wanted to learn if this would address their needs and expectations.
References
Catalog app page
New Designs
create custom app page

Adding kernel extensions + privacy preferences

In order for IT Admins to successfully deploy anti-virus software (one of the biggest reasons why customers wanted custom apps support), they'd need to additionally configure "kernel extensions" and "privacy preferences."
‍
These are technical concepts that newer IT Admins might not be familiar or have much experience with. Our point-of-view was that surfacing these options together with the package URL would provide a simple, clear mental model for deploying a custom app and its necessary settings.
kernel extensions
privacy preferences

Deploying a custom app

Once a custom app is created with any additional settings, IT Admins can assign it to groups of employees and devices. I used our existing pattern for deploying Catalog Apps–a dropdown menu and card component.

What we learned

I worked with our user researcher to develop a study plan around my learning goals, and we hosted remote user interviews over a week. I part-timed as a sidekick notetaker and detective huddling together after each session to review our "insights."

Providing a Package URL works for an MVP, but directly uploading a file would be better.

All 5 participants said self-hosting their app and providing a URL to a .pkg was "sufficient" to solve their needs today. Given a Help article, they were "confident they could figure it out." While not ideal, this was good enough signal to move forward and unblock customers as fast as possible.
‍
Testing also revealed that more experienced IT Admins expected the ability to upload an app's .pkg file directly to Fleetsmith in the future. Duly noted for our next iteration.

Admins liked seeing kernel extensions and privacy preferences together, but they didn't need to be prominent.

While 4 of 5 participants preferred having these accessible from the "Create Custom App" page, most had limited prior experience with kernel extensions and privacy preferences and didn't initially realize they were optional based on their visual prominence.

Use clearer hierarchy and language for the primary action. "What does the 'Save' button do?"

4 of 5 participants expressed some confusion around what to expect when hitting Save. "Will it save all page elements or just the title and description [above it]?" "Will it save what I have or create the app?"
‍
This invalidated my assumption that providing a familiar page structure would align with their existing mental model of apps in Fleetsmith. Instead, the Save button's placement and word choice made participants hesitant and unsure.

Changes from testing

Guided creation: 1 thing at a time

For a technical task like creating a custom app, providing a menu of too many options combined with an unclear content hierarchy tripped up folks.

To solve both issues, I introduced a single column layout that would guide users through 1 task at a time, reduce visual clutter to create focus, and repositioned the primary action atop the screen to better frame its content. I also updated the button language to clarify the intended action and made a couple other visual refinements.

Added friction for added security

"Secure by design" is one of the Fleetsmith's core values. When an Admin provides a URL to their app package, we can't verify the contents of the file. So, we required users to provide us with the package's SHA-256 as an extra layer of authentication.

To make this step as accessible to new and non-technical Admins as possible, I included a solution they could copy-paste in Terminal and ensured we covered this in our Help article.

Launch, learn, and keep going

After getting buy-in from internal reviews (yes!), we built and released this as our MVP in beta.
Takeaways
Results
The product manager and I worked with our Sales and Support teams to facilitate walk-throughs and feedback sessions with select customers. We learned how IT Admins tested the feature before rolling out custom apps to their fleet, where they stumbled, and that most were able to successfully deploy anti-virus and other custom software to their employees' devices!
Opening up user research
In our retro, I learned a couple folks were interested in hearing more user feedback earlier in the process. This was really exciting to hear as it's easy to get heads down when moving quickly! For future interviews, I put out an open invite for folks to observe the action. (1 person per session to not overwhelm our interviewees.)
Bake in longer time for QA
During the development phase, our team underestimated how long we needed to conduct feature testing. Pains of a fast growing startup–but it spurred us on to bake in more time, planning, and collaboration moving forward.

Custom Apps V2: File Upload 📦

While we were able to quickly get custom apps support in the hands of beta customers and enable them to deploy apps they needed to their employees–we could do better. Our next step was to build on our foundation, reduce the steps to deploy a custom app, and meet our customer expectation around having a direct upload capability.

Our goal: Allow IT Admins to upload a package directly to Fleetsmith and auto-calculate the SHA-256, so they don't have to.
Adding a package via file upload

I explored a few interaction patterns for selecting between providing a custom app via a package URL or file upload. I landed on using our comfy toggle component and having file upload selected by default.

When Admins upload a file directly to Fleetsmith, we would automatically scan and calculate the package's SHA-256 on the backend.

File uploader updates

This effort introduced new use cases around file uploading and management that prompted building on our simple uploader. For example, we only accepted packages in .pkg format, didn't enforce upload limits, and offered limited free storage per account.

I worked with engineers to assess what information we could extract and access in order to provide a useful experience for any situation they found themselves in.

Base Component
new states
Specifies file type
Displays file size and timestamp
Indicates progress and upload speed
Fail state with call-to-action
Not enough storage
Storage limit reached
Managing custom apps

To help IT Admins manage their custom apps, easily know which are URL vs. file-based, and keep track of their storage limit, I added subtle iconography and file size to differentiate custom app types and give them a little visual flair.

Final thoughts

If you're still with me, you seriously rock! Truly thanks for coming on this journey with me. To hear our final results after we released to GA, feel free to reach out. I'll wrap with a few thoughts on what made this project memorable and fun:
‍
From knowing nothing...
Having recently joined, I literally knew nothing about IT Administrators, what types of jobs they did, how they thought, what problems existed in their world. I had to wrap my head around an industry I had no mental model for—device management—what even is it, what higher level problem did Fleetsmith exist to solve, and how were we uniquely positioning ourselves. Then, I needed to quickly understand what this very technical feature we aimed to build was, learn the language, and guide us on a journey to make something meaningful. 💖

To knowing something
Because I jumped in knowing so little about a lot of things, I relied on collaborating with a lot of people–my technical PM and team members to understand concepts and context, our resident and ex-IT Admins to collect anecdotal war stories and build empathy, our user researcher who was just as excited as I to dive deep into the minds of our customers, Sales and Support to connect us with customers for feedback, and my design manager to help gut check my approaches with a second pair of eagle eyes. ✌️

The real treasure
We moved quickly to meet our deadline (and did!) without knowing exactly where we’d end up, but we created space to talk through what we were thinking, interrogate our ideas, share honest feedback, and have a few laughs while figuring it out together. I think that was the real treasure we found along the way. 💎
Check out another story.
Find me on
LinkedIn
,
Twitter
,
or
get in touch
. ✌️
© Lindsay Lee 2023