AI Guidelines

With AI coming onto the scene there is much debate on how to best utilize it. This represents a living document of my current thoughts

How AI works

This is a critical piece of the puzzle to understand when leveraging AI. A good primer on LLMs can be found here. People often use LLM, entire AI systems, or product names interchangeably, this often adds to the confusion. When directly interacting with LLMs (Foundry Local, Ollama, etc), you will get a better idea of what the model is providing vs what the systems around the LLM are providing. A very simple example of this is the retrieval-augmented generation (RAG) pattern. The idea here is that the LLMs responses are grounded in knowledge to provide more accurate response. It is not the LLM that has gotten better, it the system leveraging the LLM that is now able to provide more accurate information.

AI Different Usages

There are two key areas of AI usage. Using AI-based tools (Copilot, Claude, Windsurf, etc), and building solutions with AI (leveraging AI services such as Microsoft Foundry, OpenAI apis, etc).

For the purposes of this document, I will use "using AI" to mean AI based tools and "building AI" to mean building solutions with AI.

Building AI

Responsible AI

When implementing AI solutions, development teams must take this into account. Microsoft has great guidance on this.

Using AI

Guidelines

It is a popular misconception that AI can do everything.

Avoid AI when learning is desired

AI can be a great asset for producing results, however, this can come at the cost of learning. If you are trying to learn a new skill, it is often best to avoid using AI until you have a good grasp of the fundamentals. This will allow you to better understand how to leverage AI when you do start using it. This does not mean you should avoid AI entirely, but rather use it in a way that complements your learning rather than replacing it.

Do not submit code for peer review that you have not reviewed

Peer reviews (often in the form of pull requests) are first and foremost a way to ensure knowledge transfer between multiple team members. It provides opportunity for people to provide feedback and ask questions. If you have not reviewed the code yourself, do not submit it for review from others. In addition to not valuing the reviewer's time, despite you being the author of the PR, you have actually denied yourself the knowledge of the code that you are submitting.

"Correct" and "works" are not synonyms.

I would not accept someone telling me that they copied code of stack overflow with the justification of "it works", I would not accept AI generated code that has not been validated. Validation does not always need to be done by a human, but validation must occur. The validation could take for the form of tests to ensure accuracy. In the cases of UI code, running it might be the best validation. It is also worth mentioning that test coverage and validation are not synonyms, especially if tests are vibe coded. AI agents often focus on making tests pass, testing what is there rather than what is intended. If the tests are also AI generated, extra care must be taken to ensure that the test are focused on the business requirements not simply testing all of the code paths.

AI can not take blame

I have often repeated the phrase "he who takes credit also takes blame". This is a critical piece of the puzzle when using AI. If you are using AI to generate content, you are ultimately responsible for that content. If AI is allowed to autonomously build and ship code, the humans that put that process into place are to blame. This is a key detail, if the AI can not take blame, then it can not be held accountable for its actions. This means that the humans that are using AI must be responsible for the content that is generated and the actions that are taken as a result of that content. It also is unable to receive the credit that so many people are granting it.

Avoid buzz-word hype

It is popular right now to throw around every AI buzz-word in the book. As engineers we need to be exact in our speech and avoid using buzz-words that do not add value to the conversation. This is especially important when discussing AI, as there is already a lot of confusion around the topic. Using buzz-words can add to that confusion and make it harder for people to understand what is being discussed.

Velocity does not equal productivity

AI can generate huge amounts of content in a short period of time. However, this does not necessarily mean that it is productive. Sometimes the code must be tweaked or rewritten entirely. Productivity must be measured by considering defect rates, maintainability, and team velocity. AI can improve all of these, but often given way too much credit or blame for its output. It is important that we accurately measure the output of a team, and judge based on that.

Tips and tricks

Plan then Apply

This paradigm is very similar to the ideas behind Test Driven Development (TDD). Spending time focusing on building up an accurate plan for what the AI should produce will lead to significantly better results.

Critique rather than create

For important writing tasks, I favor using AI to critique my writing rather than generate it from scratch. This preserves my tone and style while leveraging AI to provide insights into how to make it better. Often people just want to have AI do all of the work, however, pivoting to having it help rather than do can lead to much better results.

Don't rely on AI when static analysis can do it

As much as possible enforce coding practices with static analysis tools. Use the output of these tools as context for the AI so that it can generate code that aligns with the standards. In .NET, there are a plethora of analyzers that can be used to enforce coding standards.