Explore Screens validation process for language models. Learn to audit, refine, and optimize standards to ensure accurate and reliable results in your workflows.
Screen Validation Process
Introduction
In this technical help document, we'll guide you through the screen validation process, an essential step in ensuring the accuracy and reliability of screens in your workflows. If you've watched our video on prompt engineering, you know the power of large language models. Now, let's dive into the validation features integrated into our app.
1. Understanding Validation
Before fully relying on a screen, it's crucial to validate its performance against your standards. Our validation process allows you to easily audit responses and track results, enabling you to update your prompts and standards as needed.
2. Creating Standards
Let's explore a sample standard to demonstrate the validation features. In this example, the standard is: "The vendor must not receive broad usage rights in data provided by the customer." We'll walk through the definitions and validation steps for clarity.
2.1 Standard Definition
The standard involves terms like "broad usage rights" and "data provided by the customer." We use AI guidance to clarify these terms, providing explicit definitions and exclusions.
2.2 AI Guidance
Our AI guidance helps define terms, such as excluding usage data from the definition of data provided by the customer. It also introduces terms like "permitted purposes" and identifies potential pitfalls, like feedback clauses.
3. Running Validation
Let's run this standard on a sample document, the DocuSign MSA, to see how it performs.
3.1 Validation Result
The screen flags the document as a fail due to the vendor's right to use feedback provided by the customer, a violation of the defined standard.
3.2 Thumbs Up/Down
To improve accuracy, you can provide feedback by giving a thumbs up or down based on whether you agree with the result. This feedback is crucial for refining and fine-tuning your standards.
4. Iterative Validation
If you encounter issues with the results, iterate on your standard. In our example, we modify the prompt to explicitly include feedback as a permitted purpose, eliminating confusion.
4.1 Clearing Results
After modifying the standard, clear the validation statistics, and rerun the screen. This ensures a clean slate for evaluating the updated standard.
4.2 Re-validation
Review the new results, and if satisfied, give a thumbs up. Continue this iterative process to achieve a highly accurate and fine-tuned standard.
Conclusion
We hope this technical help document has provided insights into the validation process. Feel free to explore and experiment with our validation features. Your feedback is essential in enhancing the performance of our app. If you have any questions or encounter issues, please let us know.
Enjoy validating your screens!