Configuring Answer Generation Settings

While Vendorful has numerous optimizations to maximize answer quality and minimize "hallucinations" operating behind the scenes, users do have the ability to tweak some parameters. In particular, when answering a questionnaire, you can guide Vendorful to give you the answers you want by making sure that you select the right content and best settings.

Generation Settings

To configure the settings that are used to answer a questionnaire, click the "Generation Settings" button on the top right of the screen when in Annotation mode. Here's what you'll see:

Content Selection

After you click that button, you'll be taken to a page where you have fine-grained control over the content. One of the keys to getting high-quality answers is to make sure that the most appropriate content is being leveraged for particular questionnaire responses. (If you haven't been labeling your content, now is a good time to revisit that.) When you open the Generation Settings, you will have the ability to choose the labels that apply to the content you want to reference.

Let's break this screen down into its components.

Select Content Labels

The first area allows you to Select Content Labels. As you start typing, suggestions will appear. You can also leverage the dropdown and see a list of labels. As content labels are selected, files with those content labels will appear in the "Referenced Content Sources" list below.

Exclude Content Labels

The second drop-down selector allows you to Exclude Content Labels from the selection. When combined with the "Select Content Labels" feature, this enables you to apply rich content selection rules in complex content libraries. For example, for a non-US InfoSec response you might include content with the "Security" label, but exclude anything with the "US" label to avoid bringing in irrelevant or incorrect information.

Select Specific Documents

In certain circumstances, you might want to include particular documents even if they aren't covered by the labels you selected. Again, you can click the dropdown or start typing and you'll be able to choose specific documents. As you add these documents, they will also appear in the list of "Referenced Content Sources."

Referenced Content Sources

While you might have added some specific documents that weren't contemplated by a selected label, you might also want to "deactivate" particular pieces of content in answering a questionnaire. On the right, under the 'Manage" column, you'll see a toggle. When a toggle is in the "on" position, you'll see some blue. When a toggle is in the "off" position, it will be grayed out.

Advanced Options

Choosing the most relevant content should make a significant impact on the generated answers. In addition to this fine-grained content selection, you can see advanced options simply by clicking the "Advanced Options" text on the Generation Settings screen. When you do that, it will expand.

Strict Relevance

When answering a question in a questionnaire, Vendorful will search your selected content for the most relevant information. By default, it gathers — where possible — a moderately broad set of content. This is particularly useful when the questionnaire contains questions you've never seen before.

Pro: By pulling together disparate information, the AI is best equipped to construct an answer.

Con: The quality of the answer may be lower because it might include less relevant information.

By selected "Strict Relevance," you are guiding Vendorful to limit the scope of relevant content to a smaller subset, and only those that reach a higher relevancy threshold. This works best when there are questions in the questionnaire that are very similar to questions and answers that you have in your content library.

Pro: By focusing more narrowly on the content that is highly relevant, the chances of a higher-quality answer improves.

Con: Restricting the information that is pulled from the content library decreases the likelihood that Vendorful will generate an answer, particularly if the question is very different from the questions in the content library.

Reference Generated Questions

Vendorful's AI Assistant spends much of its time answer questions from users. Unsurprisingly, content that is in QA format is generally the best kind of content to add. However, the system supports less structured documents as well. When the Reference Generated Questions box is checked, Vendorful will analyze non-QA sources and enrich them with hypothetical questions which are answered by the provided text. This generally improves the AI Assistant's ability to find the most contextually relevant material in documents.

Additional Instructions

If you want to provide some additional guidance to the AI about how to craft answers to questions, this is the place to do it. Another term you might hear that relates to this feature is "custom prompt." The interface for Large Language Models, the underlying AI technology used in the Vendorful AI Assistant, is prompting.

As it pertains to how best to leverage this feature in the AI Assistant, there are no hard-and-fast rules. To help get you comfortable with the feature, however, we've put together some examples that you can use for some basic experimentation.

Some practical ones.....

• Make the answer very short.
• Make the answer long and expansive.
• Answer in a style that would be pleasing to a CISO. (Note: a CISO is a Chief Information Security Officer.)

Some fun ones....

• Answer like a child.
• Answer in the form of a poem.
• Answer like a pirate.

As with the other Generation Settings, the Additional Instructions you specify (or don't specify) can be applied to an entire questionnaire or a single question that you want to answer.