Setting Up Your Pilot Group

Overview

Your deployment plan defines the phases. Your pilot group is where Phase One becomes real. The users you select, how you configure their access, how you onboard them, and how you collect their feedback will determine whether you have the data and confidence to expand.

This video covers the practical steps: who to include, how to set up the group technically, how to onboard users effectively, and how to build feedback loops that give you actionable data.

What You’ll Learn

• Selection Criteria: What makes a good pilot user and how to compose a balanced group
• Technical Setup: Creating the Entra ID group, assigning licenses, and pre-testing
• Onboarding: Welcome communications, quick-start guides, and kick-off sessions
• Feedback Collection: Surveys, channels, office hours, and what questions to ask

Script

Hook: the pilot is where the plan meets reality

Your pilot group is the proving ground for your entire Copilot deployment. The technical configuration, the training approach, the support model, the feedback you collect—everything you learn in the pilot informs every phase that follows.

Get this right and expansion becomes straightforward. Get it wrong and you’re debugging problems at scale instead of at pilot size.

Let’s set up your pilot group for success.

Selection criteria and composition

Start with who you’re selecting and why.

The most common mistake is filling your pilot with enthusiasts—people who are already excited about AI and will love Copilot no matter what. That gives you positive feedback but not useful feedback. You need a group that represents the diversity of your organization.

Digital literacy is the baseline requirement. Pilot users should be comfortable with Microsoft 365. They use Teams for communication, SharePoint for documents, and Outlook for email. They don’t need to be power users, but they can’t be struggling with the basics. Copilot builds on M365 proficiency—if someone doesn’t use Teams, they can’t evaluate Copilot in Teams.

Feedback willingness matters as much as technical skill. You need people who will actually try Copilot regularly, report what works and what doesn’t, and respond to surveys. Ask managers to nominate people who are reliable communicators, not just technically capable.

Role diversity is critical. Include program staff who write reports and analyze policy. Include administrative support staff who manage calendars and correspondence. Include technical staff who work with data and systems. Include at least one or two leaders who attend meetings and make decisions. Each role uses Copilot differently, and you need to see all of those patterns.

Include skeptics. People who are cautious about AI, who wonder whether Copilot will actually help, or who have concerns about data privacy. If skeptics become advocates during the pilot, that’s the most powerful signal you can send to the rest of the organization.

Make sure you have representation across departments and, if applicable, locations. A pilot that only includes headquarters staff won’t tell you how field offices experience Copilot.

Target size: 50 to 200 users. Below 50, you don’t get enough diversity of use cases. Above 200, you’re managing a small deployment, not a pilot. The sweet spot depends on your organization’s size, but 100 users is a good default.

Technical setup

With your pilot list finalized, here’s the technical setup.

Create a dedicated security group in Entra ID. Name it clearly—something like “Copilot Pilot Phase 1” so there’s no confusion about its purpose. This group will control Copilot license assignment and can also be used for targeting Conditional Access policies, communications, and reporting.

Assign the Microsoft 365 Copilot license to this group using group-based licensing. Go to the Entra admin center, select the group, navigate to Licenses, and assign the Copilot license. This way, managing pilot membership is as simple as adding or removing users from the group.

Before you add all pilot users, test with two or three members of your IT team. Add them to the group, wait for license propagation, and verify that Copilot appears in their apps. Test across Word, Teams, Outlook, and Excel. Confirm that audit logging is capturing Copilot events. Run through your Conditional Access policies using the “What If” tool to make sure everything behaves as expected.

Once you’ve validated with your test users, add the full pilot group. Stagger if needed—add users in batches of 20 to 30 over a day or two rather than all at once. This helps you catch any issues early without affecting the entire pilot group.

Verify that all pilot users are on supported Microsoft 365 app versions. If anyone is on an older build, push an update before the pilot kick-off. Copilot won’t appear on unsupported versions, and you don’t want your first pilot interaction to be a troubleshooting session.

Onboarding your pilot users

Technical setup gets Copilot to your users. Onboarding gets your users to Copilot.

Start with a welcome communication. Send an email or Teams message to every pilot user explaining three things: why they were selected, what Copilot is and what it can do, and what you need from them. Keep it concise—one page, not a manual. Make them feel valued, not burdened.

Provide a quick-start guide. This should be a simple document—two pages maximum—covering where to find Copilot in each app, how to write a basic prompt, three to five example prompts they can try immediately, and what to do if something doesn’t work. Don’t try to make them experts. Give them enough to get started and explore.

Hold a kick-off session. This is a 30 to 45 minute live session where you demonstrate Copilot, show it working with real organizational content, and answer questions. Do this within the first week of the pilot. Let people see it in action before they try it on their own.

Set expectations clearly. This is a learning experience, not a finished product launch. Things might not work perfectly. Some features might not be available in your government cloud. That’s why you’re piloting—to find these things before you roll out broadly.

Finally, provide a direct support channel. Create a dedicated Teams channel for the pilot group where they can ask questions, share tips, and report issues. Staff it with someone from your IT team who can respond within a business day. Users who hit a wall and can’t get help will stop trying Copilot.

Feedback collection mechanisms

Feedback is the entire point of the pilot. Set up your collection mechanisms before the pilot starts, not after.

Structured surveys are your primary tool. Send a survey at the two-week mark and another at the four-week mark. Keep them short—10 questions maximum. Ask about frequency of use, perceived usefulness, specific features they’ve tried, pain points, and whether they’d recommend Copilot to colleagues. Use a consistent rating scale so you can track trends.

A dedicated Teams channel gives you real-time, unstructured feedback. Encourage pilot users to post observations, questions, and tips as they use Copilot. This captures insights that surveys miss—the “I tried this and it was amazing” moments and the “I expected this to work but it didn’t” frustrations.

Weekly office hours or drop-in sessions give users a chance to ask questions and share experiences in a live setting. These are especially valuable in the first two weeks when users are forming habits. Keep sessions to 30 minutes and make them optional—mandatory attendance kills the collaborative spirit.

Usage data from your admin reports fills the quantitative gap. Track how many pilot users are actively using Copilot, which apps they’re using it in, and how frequently. Compare this to your survey responses. If someone says they love Copilot but the usage data shows they’ve used it twice, there’s a disconnect worth exploring.

Individual check-ins round out your feedback picture. Talk to your power users—what are they doing that works? Talk to non-users—why aren’t they engaging? These conversations reveal adoption barriers that surveys and usage data can’t capture.

What to ask across all channels: Is Copilot useful for your daily work? How often are you using it? What works well? What doesn’t? What surprised you? Would you want to keep it if the pilot ended?

Close: from pilot to decision

At the end of your pilot period, synthesize everything.

Map your feedback against the success criteria you defined in your deployment plan. Did you hit your active usage targets? Are satisfaction scores above your threshold? Did you encounter any security incidents or compliance concerns?

Build the case for expansion with real data. Don’t tell leadership “users liked it.” Tell them “73 percent of pilot users actively used Copilot weekly, satisfaction averaged 4.1 out of 5, and the most common request was to expand access to their teams.”

Document your lessons for governance records. What configuration changes did you make during the pilot? What training gaps did you identify? What issues did you resolve? This documentation supports your next deployment phase and provides evidence for your authorizing official.

Your pilot isn’t just a test—it’s your evidence base. Make it count.

Sources & References

• Microsoft Copilot adoption resources — Adoption framework and pilot planning
• Enable users for Microsoft 365 Copilot — User enablement guidance
• Microsoft 365 Copilot setup — Setup and deployment guidance