Danodocs | AI

Prediction Guard - Semantic Layer

Wed, 11 Mar 2026 13:26:44 -0600

High Level Notes

DA: seems like the core concept was how to create a “semantic layer” between a user facing LLM or Agent and an enterprise’s underlying data (data lakes, OLTP databases, traditional documents, etc.)
DA: will post slides if/when I get them
- Dan written notes from the webinar: Download DA’s notes

Three Principles of Semantic Layering

Principle #1 | Data Curation

Don’t unleash all of your enterprise data on the Agent. Curate key subsets of the data and surface that data to the Agent world

Notes & Code Snippets

Wed, 25 Feb 2026 18:50:03 -0600

View this page on GitHub

OpenAI Chat Completion Request

 client = OpenAI(api_key="<OPENAI_API_TOKEN>")

# Create a request to the Chat Completions endpoint
response = client.chat.completions.create(
 model="gpt-4o-mini",
 messages=[{"role": "user", "content": "In the context of marketing for a new Italian family-style restaurant, generate a marketing slogan to be used in marketing materials including a website and print."}],
 max_completion_tokens=100
)

print(response.choices[0].message.content)

Classification

Zero Shot

Using a “Zero Shot” prompt (don’t include any examples in the context), generate a sentiment classification of product reviews.

Notes & Code Snippets

Wed, 25 Feb 2026 18:50:03 -0600

Notes

Vibe Coding Notes

Sun, 22 Feb 2026 17:55:03 -0600

Links & Resources

Adal Coding Agent: https://sylph.ai/
Zach Wilson’s Vibe Coding Bootcamp:
- Day 1: https://youtube.com/live/BSN0uwavP4I
- Day 2: https://www.youtube.com/live/pxUMU755uaI

Adal Prompts

Prompts I have used

ChatGPT | Agentic AI Payment Protocols

Fri, 23 Jan 2026 13:55:03 -0600

Here’s a clear overview of the major agentic payment standards emerging in 2025–2026, the current state of adoption, and official sources/links you can consult for each.

🧭 1. Agentic Commerce Protocol (ACP)

What it is: An open commerce standard for AI agents to complete purchases on behalf of users, focused on checkout and payment interactions using existing payment rails (credit/debit cards, tokenization). It defines how agents discover products, invoke secure checkouts, and pass scoped payment tokens without exposing raw card details. (OpenAI Developer Docs)

Bootcamp Notes

Tue, 21 Oct 2025 17:55:03 -0600

View this page on GitHub

Setting Up Python Environment

Use a Python 3.11 environment - later code samples will not support 3.13

 # navigate to your project folder
cd your-folder
# set up a Python 3.11 virtual environment in the `venvs` folder in your home folder: `~/venvs/py311env`
/opt/homebrew/opt/python@3.11/bin/python3.11 -m venv ~/venvs/py311env

# activate the virtual environment for this session - so python modules will be installed here (for the project)
source ~/venvs/py311env/bin/activate

# install Python packages
pip install -r requirements.txt

From the repo readme. Start the web application

Glossary

Wed, 19 Feb 2025 15:11:34 -0600

View this page on GitHub

General Links and Resources

Zach Wilson / DataExpert.io Github “Handbooks”
Chip Huyen AI Engieering Book Companion Site: https://github.com/chiphuyen/aie-book
Thomas Yeh: https://www.byhand.ai substack and online courses
- you are subscribed to his/their Substack newsletter
- He has a free Agentic AI intro course: https://www.byhand.ai/p/introduction-to-agentic-ai
- Check out the course you have joined here: https://community.genai.works/
- MEAP Book (writing in progress) https://www.manning.com/books/build-a-multi-agent-system-from-scratch

check_circle

NOTE: This should be the link to see the content of the free Agentic AI course: https://community.genai.works/spaces/18408418/content

L402.org: AI Agent payment protocol (?)
https://docs.nvidia.com/nemo-framework/index.html: NVIDIA Nemo Framework. From DataExpert.io course. Use to implement Guardrails?
MCP Integration
- https://modelcontextprotocol.io
- https://developers.openai.com/api/docs/mcp
Agent Skills or Skills: https://agentskills.io

– A –

A2A: Agent to Agent interactions/transactions - example: A research agent pays a summarization agent
- related:
  - H2A: Human to Agent - example: User pays ChatGPT or invokes a tool
  - A2H: Agent-to-Human - example: Agent pays a contractor or requests approval
  - A2S: Agent-to-Service - example: Agent pays an API or SaaS endpoint
  - B2B / P2P: traditional payments (non autonomous)
- A2A systems require:
  - Machine-verifiable identity
  - Programmatic authorization
  - Instant or near-instant settlement
  - Micropayments
  - Deterministic execution
  - Cryptographic guarantees
Agent Skills: A simple, open format for giving agents new capabilities and expertise. Agent Skills are folders of instructions, scripts, and resources that agents can discover and use to do things more accurately and efficiently
- Difference between Skills & Tools
  - Tools are the low-level primitives an agent can call (APIs/functions like “read file,” “send message,” “run shell command,” “web search”). They’re usually narrow, capability-oriented, and runtime-provided.
  - Agent skills are higher-level packages of behavior that use tools (and add procedure, guardrails, prompts, code, tests, configs) to reliably accomplish a task end-to-end (like “triage inbox,” “generate social post + schedule,” “audit SSH hardening”).
  - In short: tools = what the agent can do; skills = how the agent should do a specific job (often by orchestrating tools).
Agentic AI: Yeh Course - Agentic AI takes artificial intelligence beyond simple prompt-based responses. Unlike traditional models, agents can observe, reflect, use tools, plan ahead, and even collaborate—just like a human problem solver
Agentic AI - Payment Protocols: (relatively) early protocols addressing different types AI Agent payment use cases. Listed below are major protocol alternatives:
- Agentic Commerce Protocol (ACP): An open commerce standard for AI agents to complete purchases on behalf of users, focused on checkout and payment interactions using existing payment rails (credit/debit cards, tokenization)
  - https://developers.openai.com/commerce/
- Agent Payments Protocol (AP2): An open standard for secure, interoperable agent-initiated payments, focused on authorization, trust, and auditability of payments initiated by AI agents. AP2 uses cryptographically signed “mandates” to ensure user intent and accountability for transactions
  - https://ap2-protocol.org
- x402: Internet-Native Payments Standard: An HTTP-native payment protocol that revives HTTP 402 (“Payment Required”) to enable autonomous payments between clients/agents and services. When a service requires payment, it responds with 402 and machine-readable payment instructions; the client pays and retries
  - https://www.x402.org
- Universal Commerce Protocol (UCP): An emerging standard. A broader open commerce standard announced by Google and partners (2026) designed to unify multiple protocols across discovery → purchase → post-purchase, aimed at interoperability between agents and commerce endpoints. It complements AP2, ACP, A2A, and MCP rather than directly replacing them
  - https://www.x402.org/?utm_source=chatgpt.com
- See: so ChatGPT Response
AI Engineering: process of building applications leveraging readily available models
activation function: activation functions prevent models from grounding neural network models down to linear models - ie. they introduce non linear dynamics (DA: otherwise the model just defaults to a weighted sum plus bias expression [?])
ASR: Automatic Speech Recognition

– B –

Backpropagation: DA | a training approach to iteratively adjust weights and bias values using an error correction algorithm such as gradient descent

– C –

Chain of Thought (COT): explicitly asking the model to think step by step (“think step by step”, “explain your decision”)
- used for complex reasonging task
Chaining: breaking down prompts or model interactions into serial steps (e.g. as opposed to one encompassing prompt)
Completion: the output of an LLM that is result of processing a prompt (e.g. Go is...) with a completion of a programming language...
Context Window: how big your prompt can be that is input to the model
- Most models have limits of ~2000 tokens
Cosine Similarity: From ChatGPT Cosine Similarity is a measure of similarity between two vectors in a high-dimensional space, commonly used in LLMs for text embeddings and semantic search. It calculates the cosine of the angle between two vectors, where 1 means identical, 0 means unrelated, and -1 means completely opposite

Concept	Description	Operator/Syntax	Notes
Cosine Similarity	A measure of how one vector’s direction is similar to another vector’s direction. In Postgres `pgvector` you get this by subtracting the `Cosine Distance` from `1`	pgvector: `1 - (A <=> B)`	Values: `-1` to `1` (higher value means most similar)
Cosine Distance	Also a measure of how one vector differs from another in terms of its direction	pgvector: `A <=> B`	Values: `0` to `2` (lower value means most similar)

– D –

dspy: dspy (short for Declarative Self-Improving Language Models) is an open-source Python framework for building self-improving LLM applications.
- It allows engineers to define AI tasks declaratively and then automatically optimize, test, and refine the prompts, parameters, and reasoning strategies that an LLM uses to complete those tasks
- High-Level dspy Workflow
  1. You define a task module/signature.
  2. You provide a dataset of (input → correct output) examples.
  3. You choose an optimizer (MIPRO, BootstrapFewShot, COPRO, etc.).
  4. dspy tests many versions of system prompts internally.
  5. dspy selects the best-performing prompt/module configuration.
  6. You freeze/export the optimized program for production use.

– E –

embeddings: vector that aim to capture the meanings of the orginal text (from which the embedding was calculated)
- From ChatGPT An embedding is the dense vector form that represents a concept in a model’s “understanding space"
  - embeddings encode meaning rather than direct word identity.

🧠 3. Summary of Sparse Vector vs. Embedding

Feature	Sparse Vector	Dense Vector / Embedding
Size	= vocabulary size (can be 10k–1M+)	Typically 256–4096
Meaning of each position	Specific word or feature	Learned, abstract dimension
Zeros	Mostly zeros	Mostly non-zero
Captures semantics?	No	Yes
Memory efficiency	Poor	High
Used by	TF-IDF, one-hot, bag-of-words	Neural nets, LLMs, embeddings

In short:

x402 | Conceptual Flow

Fri, 23 Jan 2026 13:55:03 -0600

0️⃣ Links to Check Out

The DEFINITIVE guide to x402 (ft. Erik Reppel) - Coinbase Developer Podcast

1️⃣ A2A (Agent-to-Agent) — The Intent Layer

A2A answers: Who is talking to whom, and why?

MCP Servers & Governance

Fri, 30 Jan 2026 13:55:03 -0600

View this page on GitHub

Expanding on notes captured on the Glossary page

Structured Outputs

Mon, 09 Mar 2026 15:32:35 -0500

View this page on GitHub

Structured Outputs - Notes

Generating a Table

 client = OpenAI(api_key="<OPENAI_API_TOKEN>")

# Create a prompt that generates the table
prompt = """
Identify the ten "must read" science fiction books to be displayed on the homepage of a leading online bookstore with an extensive collection of science novels.

Output the list of books in a table with ten rows representing each book and include these columns: `Title`, `Author`, `Year`. Output ten books in the generated table.
"""

# Get the response
response = get_response(prompt)
print(response)

Using format strings to build prompts

 client = OpenAI(api_key="<OPENAI_API_TOKEN>")

# Create the output format
output_format = "Output three separate lines which will display the text copy, language detected, and the title generated. Those lines should be prefixed with `Text:`, `Language:`, and `Title:` respectively."

# Create the overall instructions referencing variable components
instructions = f"""
Analyse the text listed below delimited by triple backticks. The analysis should detect the language of the delimited text and generate a short title for the text content.

The text to analyze is:
```
{text}
```

Generate tht output using the following format:
{output_format}
"""

# Create the final prompt
prompt = instructions.format(text=text, output_format=output_format)
response = get_response(prompt)
print(response)

Using conditional prompt language

 client = OpenAI(api_key="<OPENAI_API_TOKEN>")

# Create the instructions
instructions = "Analyze the text listed below delimited by triple backticks. Determine the language of the text and the number of sentences in the text snippet. If the text snippet consists of more than one sentence then generate an appropriate short title for the text paragraph, otherwise generate a title string of `N/A`."

# Create the output format
output_format = "Output the analysis as separate lines. Output a line displaying the text snippet prepended by the string `Text:`, a second line displaying the identified language prepended by string `Language:` and a third and final line displaying the generated title prepended by string `Title:`"

prompt = instructions + output_format + f"```{text}```"
response = get_response(prompt)
print(response)

 client = OpenAI(api_key="<OPENAI_API_TOKEN>")

# Create the instructions
# Use analogous if-then-else language to define a conditional prompt element
instructions = "Analyze the text listed below delimited by triple backticks. Determine the language of the text and the number of sentences in the text snippet. If the text snippet consists of more than one sentence then generate an appropriate short title for the text paragraph, otherwise generate a title string of `N/A`."

# Create the output format
output_format = "Output the analysis as separate lines. Output a line displaying the text snippet prepended by the string `Text:`, a second line displaying the identified language prepended by string `Language:` and a third and final line displaying the generated title prepended by string `Title:`"

prompt = instructions + output_format + f"```{text}```"
response = get_response(prompt)
print(response)

“Chain of Thought” Prompting

Prompt example

Assignment #1

Mon, 10 Nov 2025 17:55:03 -0600

View this page on GitHub

Homework #1

Homework #1 Submission zip file

Prompt Engineering

 # Most Frequent Character in a Country Name

This analysis identifies the country whose official name contains the most frequently repeated letter. 
Because “official” country lists vary across sources, the dataset used here is the **United Nations list of sovereign states**, as published on [Wikipedia](https://en.wikipedia.org/wiki/List_of_sovereign_states#List_of_states).

The following Python script counts letters in each country name (case-insensitive, ignoring spaces and punctuation) and reports the country with the single most frequently repeated character.

```python
def char_count(name: str):
 """
 Return (most_frequent_char, count, error_or_none) for a given country name.
 Counts letters only, case-insensitive.
 """
 if not isinstance(name, str) or not name:
 return None, 0, "err: passed an empty string or non-string value"

 counts = {}
 for ch in name.lower():
 # Use .isalpha() to ignore spaces, commas, etc.
 # Remove this check if you want to count all characters
 if ch.isalpha():
 counts[ch] = counts.get(ch, 0) + 1

 if not counts:
 return None, 0, "err: no countable characters"

 # Pick the (char, count) pair with the highest count
 most_char, max_count = max(counts.items(), key=lambda kv: kv[1])
 return most_char, max_count, None


def main():
 countries = [
 "Afghanistan", "Albania", "Algeria", "Andorra", "Angola",
 "Antigua and Barbuda", "Argentina", "Armenia", "Australia",
 "United States of America", "Uruguay", "Uzbekistan", "Vanuatu",
 "Venezuela, Bolivarian Republic of", "Viet Nam", "Yemen",
 "Zambia", "Zimbabwe"
 ]

 max_country = None
 max_char = None
 max_count = -1

 for country in countries:
 ch, cnt, err = char_count(country)
 if err is None and cnt > max_count:
 max_country, max_char, max_count = country, ch, cnt

 print(
 "Most frequently repeated character\n"
 f"Country: {max_country}\nChar: {max_char}\nNum: {max_count}"
 )


if __name__ == "__main__":
 main()
```

## Output

The script identified **“United Kingdom of Great Britain and Northern Ireland”** as the country name with the most repeated letter. 
The letter **“n”** appears **seven times**.

```
Most frequently repeated character
Country: United Kingdom of Great Britain and Northern Ireland
Char: n
Num: 7
```

---

# Prompt & Model Experimentation

To evaluate how different LLMs handled this question, I tested multiple prompts and models using the OpenAI API. 
The experiment compared various phrasing strategies and model versions to measure accuracy and consistency.

## Models Tested

| Model | Accuracy | Required Source Prompt | Notes |
|--------|-----------|-----------------------|--------|
| GPT-4 | ❌ Often incorrect | ✅ Yes | Miscounted or inconsistent results |
| GPT-4o | ❌ Similar to GPT-4 | ✅ Yes | Slightly improved consistency |
| GPT-5 | ✅ Correct | ✅ Yes | Matched expected answer consistently |

## Prompt Characteristics

1. A simple text prompt asking the core question. 
2. Extended prompt instructing to treat vowels and consonants equally, and include multi-word names. 
3. Further extension emphasizing inclusion of stop words (e.g., *the*, *of*, *and*). 
4. More detailed instructions including a step-by-step task list. 
5. Prompts directing the model to use the **United Nations Member States** list as the official country source.

---

# Discussion on Results

### Performance by Model

Older models such as GPT‑4 and GPT‑4o performed inconsistently. In most cases, they produced incorrect results or miscounted letters. 
GPT‑5, by contrast, returned accurate and consistent results—especially when explicitly prompted to reference the UN Member States list.

### Why GPT‑5 Succeeded

The GPT‑5 model responded correctly across multiple prompt variations. 
The most reliable answers came from prompts that:

* Used GPT‑5 
* Included a link to the UN website as the authoritative source 
* Clearly explained how to count letters, including conjunctions and prepositions 

This suggests that GPT‑5’s performance benefits from both precise task instructions and explicit grounding in a definitive dataset.

---

# Successful Prompts and Responses

Below are selected prompt–response pairs in JSON format.

```json
[
 {
 "model": "gpt-4",
 "category": "g-promptLetterDescMinorWordsTaskList-WithSource",
 "prompt_text": "In the context of world geography, can you tell me what country has the same letter repeated the most in its name?...",
 "prompt_resp": "From my training data, the longest country name is 'The United Kingdom of Great Britain and Northern Ireland'..."
 },
 {
 "model": "gpt-5",
 "category": "b-promptSimple-WithSource",
 "prompt_text": "In the context of world geography, can you tell me what country has the same letter repeated the most in its name?...",
 "prompt_resp": "Short answer: United Kingdom of Great Britain and Northern Ireland..."
 }
]
```

---

# Summary

The country name **“United Kingdom of Great Britain and Northern Ireland”** contains the most frequently repeated letter (**n = 7**) among UN‑recognized sovereign states. 
Across multiple model generations, GPT‑5 consistently produced the correct result when given detailed instructions and a definitive country list source.

`main.py`

Update the /api/parse-resume route handler

AI Applications

Sat, 22 Feb 2025 18:36:40 -0600

Planning AI Applications

From the book AI Engineering

Use Case Evaluation

Question: why do you want to build this application?

Competitive Pressure: if you don’t build will competitors with AI driven capabilities make you obsolete?
Leverage Bigger / New Opportunities: if you don’t build will you miss opportunities to drive increased revenue and profit; take advantage of new opportunities
Understand New Technologies: if not sure how AI impacts the business, at least get an understanding of the impact

Think about the build vs buy paths

Assignment #2

Sun, 23 Nov 2025 17:55:03 -0600

View this page on GitHub

Homework #2

Homework #1 Submission zip file

Brief Notes

See the zip file for the major code deliverables for this assignment including:

Assignment #3 - Capstone Project Proposal

Sun, 23 Nov 2025 17:55:03 -0600

View this page on GitHub

Homework #3

Homework #3 Capstone Proposal Submission zip file

Brief Notes

See the zip for the the proposal text (.md) and system context diagram

Zach AI Agent feedback

The feedback is detailed and may be helpful in the future

Prompt Engineering

Wed, 25 Feb 2026 18:50:03 -0600

View this page on GitHub

Notes

Principles of Prompting

Use Action verbs
- Write
- Complete
- Explain
- Generate
- Describe
- Evaluate
- …
Formulate detailed instructions: provide specific, descriptive, and detailed instructions on:
- Context
- Output length
- Format & style
- Audience

Basic OpenAI API Code

Stub code to generate a response from OpenAI

Neural Networks

Sun, 23 Feb 2025 15:55:17 -0600

Perceptrons

A perceptron is a 1 neuron (node) model tha can be used to solve certain problems

Basic examples are modeling: AND gate, OR gate
Perceptrons work for types of problems in which linearly separable, that is there is linear expression that clearly separates outcomes.
- in other words you can draw a line and the different types (e.g classes) of outcomes are separated

Intuition

the point of the weighted neuron and bias sum calculation is to define the line that serves as the decision boundary
all points on one side of the decision boundary are in one class (e.g. 1) and the others are in the other class (e.g. 1)
an activation function (threshold function) calculates the output node (which should convert the calculation over or under the decision boundary)
Process
1. configure an architecture (2-2-1) of:
  - 2 input neurons (x to be ANDed with y)
  - a “hidden” layer of 2 neurons that serve as the interim output of the weigted sum & bias calculation
  - 1 output neuron

A perceptron computes a weighted sum of its inputs:

Advanced Prompt Engineering

Mon, 09 Mar 2026 15:32:35 -0500

View this page on GitHub

“Shot” Prompting

Few Shot prompting

Large Language Models

Wed, 19 Feb 2025 15:34:22 -0600

Resources and Links

Dan’s Lambda School DS Youtube Playlist
- (probably dated but maybe good background on certain topics?)
Daniel Whitenack’s (DW) go-genai-webinar repo
- seminar presentation: link
- Youtube seminar video (sponsored by Arden Labs): Youtube link
Prediction Guard Documentation - some open models available
Cohere: https://cohere.com/ | Cohere brings you cutting-edge multilingual models, advanced retrieval, and an AI workspace tailored for the modern enterprise — all within a single, secure platform
DW suggested links:
For building personal / low expense outlay look at HuggingFace (ZeroGPU program?)
DeepNote: AI driven notebooks - https://deepnote.com/
Data Science from Scratch code: https://github.com/joelgrus/data-science-from-scratch

Notes on Large Language Models

(Very) High Level Notes on LLM Execution
- LLMs take a prompt and then calculate probabilities of words (tokens?) that should follow each other
- For prompt Go is... the LLM may generate these words in descending order of probability:
  1. a
  2. programming
  3. language
  4. …
- NOTE: this is very similar to autocomplete
- LLMs will calculate probabilities for every word (token?) it knows about (DA: scale seem massive)
- LLM Temperature configuration setting: sounds like it drives some sort of variablity into the output so that the results are not always driven by strict probabilites (this would be boring or tend to lack creativity)
Enterprises are gradually migrating to use Open LLMs (e.g. Llama 3, deepseek, Mistral) from Closed LLMs (e.g. OpenAI [ChatGPT], Anthropic [Claude])
Daniel Whitenack’s spectrum of AI complexity.
- Basic Prompting
- Prompt Engineering (CoT, templates, parameters)
- Augmentation, Retrieval
- Agents, Chaining
- Fine-tuning via a closed API
- Fine-tuning an open model
- Training a model from scratch
Prompts
- LLMs are tuned to use prompts that are formatted in specific ways (rather than a simple text question)
  - Check out: https://docs.predictionguard.com/guides-and-concepts/using-ll-ms/prompt-engineering
Accuracy
- Autocomplete LLMs focus on coherency so they answers may be coherent but not necessary accurate. PredictionGuard uses factual consistency checking models to confirm accuracy.
  - For example: The White House is painted pink. (the sentence is coherent not accurate)
Training Models
- DW: “You should never, ever, ever have to train a model for the rest of your life”
  - You should…
    - Use an open model and inject your data into the prompt
    - At most, you may need to fine tune a model

LLM Model API Behavior

Daniel Whitenack: most APIs including PredictionGuard (Daniel’s company) will start streaming completion text immediately and essentially stream/spit it out serially
- NOTE: In the PredictionGuard go code, they use a channel to receive that stream (and then your go program can start print it?)

Embeddings / Vector Representations

See https://cohere.com/ for a service to generate / capture embeddings

Prompts

Prompt Formatting: various models are trained to handle prompts with specific text formatting. Structuring prompts in this way should optimize execution (?)

ChatML Format

the actual text would go into the curly brace area

 <|im_start|>system
{prompt}<|im_end|>
<|im_start|>user
{context or user message}<|im_end|>
<|im_start|>assistant<|im_end|>

Large Language vs. Foundation Models

From ChatGPT Summary of differences between Large Language and Foundation Models

Understanding Foundational Models

Mon, 24 Feb 2025 17:55:03 -0600

Training Data

an AI model is only as good as the data it was trained on
common sources of general training data
- Common Crawl: sporadic crawls of web data done by a nonprofit organization https://commoncrawl.org/
  - Google has a clean subset of this data called: Colossal Clean Crawled Corpus (or CA)
general purpose foundation models will typically perform less well on domain specific tasks (due to less training data)

Modeling

What should modelers consider?
- Model architecture?
- Number of parameters?

Model Architecture

Transformer Architecture see glossary is currently the dominant archiecture for language based Foundation Models
- training is based on the attention mechanism
- intended to solve some prevailing problems:
  1. previous seq2swq architecture generated output only on the final hidden state of input (analop: like generating answers about a book by just reading it’s summary)
  2. using the RNN encoder and decoder meant that input processing and output generation is done sequentially - slow for inputs with lots of tokens
Transformer Architecture Inference: leverages a parallel nature of execution
1. Prefill step: model processes input tokens in parallel
  - this produces key and value vectors for all input tokens
2. Decode step: decode generates one output token at a time
Attention Mechanism
- use key, value, and query vectors
- From ChatGPT: the key vectors are used for matching and weighting (determining “where to look”), while the value vectors provide the substantive information (“what to show”) during the attention computation
- From ChatGPT: query vector is a high-dimensional representation derived from an input element (e.g., a token in a sentence). It represents the “question” that the model asks of other tokens in the sequence
- attention mechanism computes how much attention to give an input token by performing a dot product between a query vector and its key vector

Model Size

Number of parameters is usually appended to the model name: e.g Llama-13B ~ 13 billion parameters
Generally more parameters means more capacity to learn
- However newer models generally perform better even if they are smaller
- a parameter usually stores 2 bytes or 16 bits
a sparse model has a lot of zero value parameters
Training size
- dataset sizes are measured by the number of training samples
- Language Models: a training sample can be a sentense, Wikipedia page, chat conversation, a book
- currently LLMs are trained on datasets representing trillions of tokens

Post Training

Refine trained models: e.g. move from text completion to conversation
Post training - high level steps
1. Supervised finetuning (SFT): fine tune model on high quality instruction data to optimize models for conversations instead of completion
2. Preference fintuning: further finetune the model to output responses that align with human preference - typically using reinforcment learning
Pretraining focus on optimizing token-level quality. Post-training focuses on the quality of the overall response

Infererence

When a production LLM receives a string prompt, it follows these high-level steps to generate an inference:

Agentic AI

Mon, 31 Mar 2025 17:55:03 -0600

(from Yeh course unless specified otherwise)

Links & Resources

Frameworks
- Auto Jam – Simplifies complex AI workflows using pre-built templates.
- Meta GPT & Crew AI – Enable highly customized multi-agent simulations, mimicking human roles
- Wikipedia on multi agent systems
Apps / Tools
- Manus: https://manus.im
  - Mentioned in Yeh’s course
- Crew AI: https://www.crewai.com
Agents & Payments
- https://x402.org
  - ChatGPT suggested x402 learning project: https://chatgpt.com/s/t_6973ee78d9b48191bebe1922844b5771
- https://blockrun.ai

Background & Intro

Four behaviors of Agent behavior (from Yeh course)
1. Reflection: Before responding, the agent assesses whether it needs more information.
2. Tool Use: Agents access external resources, like checking live flight prices or retrieving updated data.
3. Planning: They break down complex tasks into step-by-step solutions.
4. Multi-Agent Coordination: Multiple agents work together like an efficient team, each handling different roles.

Yeh’s Framework (or “Equation”)

Yeh breaks down the Agent’s engagement and interaction into parts of a framework:
- See: assume this is the ability for the Agent to see (process?) an initial prompt entered by the user
- Think: assume this is a directive to the agent to engage in a particular way:
  1. Role: adopt a role as the Agent (“act as a helpful real estate agent”)
  2. Plan: step through a series of process steps to ultimately achieve the goal
- Remember: use data to aid in the process, seems to come in a couple flavors:
  - Historical Data: assume this is past interactions the Agent has had with this use (?)
  - Contextual Data: assume this is application data (e.g. from a database or system) or third party data (e.g. stock prices)
- Can: assume this is to execute on particular tasks or invoke other entities to execute tasks on the Agent’s behalf (e.g. initial a call, book an appointment)
That whole package above ⬆️ is considered a big prompt to be executed by an LLM

View this page on GitHub

Cook/Kitchen AI Model Analogy

Tue, 04 Nov 2025 17:55:03 -0600

View this page on GitHub

The Full Kitchen Story

AI Concept	Kitchen Analogy	What’s Really Happening
Model Architecture	The recipe book (list of steps: whisk, bake, reduce…)	Fixed sequence of layers (Linear → ReLU → Attention…)
Weights (θ)	Muscle memory & knob settings on ovens, mixers, timers	Learned parameters that transform inputs
Training Data	Thousands of ingredient bags (flour, sugar, spices, labeled “good cake” or “burnt”)	Labeled examples $ (x, y) $
Forward Pass	Following the recipe step-by-step to produce a cake	$ ( \hat{y} = f(x; \theta) ) $
Loss Function	Taste test by a picky judge (score 1–10)	$ ( \mathcal{L}(\hat{y}, y) ) $
Backpropagation	Judge writes notes on every step: “Too much salt here → reduce shaker next time”	Chain-rule gradients $ ( \partial\mathcal{L}/\partial\theta ) $
Optimizer (Adam, SGD)	Sous-chef who physically adjusts every knob based on judge’s notes	$ ( \theta \leftarrow \theta - \eta \cdot g ) $
Epoch	One full day of baking dozens of cakes, tasting, adjusting, repeat	Full pass over dataset
Validation Set	Separate table of guest tasters who never give adjustment notes	Monitor generalization
Overfitting	*Chef memorizes exactly* how the training cakes tasted**, fails on new guests	High training acc, low val acc
Regularization (Dropout, Weight Decay)	Randomly turning off one burner or fining the chef for using too much butter	Prevent over-reliance on any step
Learning Rate	How boldly the sous-chef turns the knobs — too big → overshoot, too small → stuck	Step size η
Gradient Clipping	Putting a safety cap on the gas valve so it can’t explode	Prevent exploding gradients

Training Phase = Apprenticeship in the Kitchen

Day 1: Open recipe book, set all oven dials randomly.
Bake a cake (forward pass).
Judge tastes → “6/10, too dry, not sweet enough” (loss).
Judge circles mistakes on every step → backprop.
Sous-chef tweaks every dial a tiny bit → optimizer step.
Repeat for 100 cakes (one epoch).
End of week: Chef now bakes training cakes at 9.8/10.
Guest table (validation) still says 7/10 → overfitting.

Inference Phase = Restaurant Service

AI Step	Kitchen Action
Load model	Open restaurant, recipe book is now laminated — no edits!
Preprocess input	Customer orders “chocolate cake” → measure exact 200 g flour, 150 g sugar (tokenize, normalize)
Forward pass	Follow recipe exactly → mix, bake, cool
Post-process	Dust with powdered sugar, plate nicely (softmax → argmax, detokenize)
Return result	Serve cake in < 2 minutes — customer happy

No judge. No notes. No knob-twiddling.
Just perfect, repeatable execution.

Danodocs | AI

Prediction Guard - Semantic Layer

High Level Notes link

Three Principles of Semantic Layering link

Principle #1 | Data Curation link

Notes & Code Snippets

OpenAI Chat Completion Request link

Classification link

Zero Shot link

Notes & Code Snippets

Notes link

Vibe Coding Notes

Links & Resources link

Adal Prompts link

ChatGPT | Agentic AI Payment Protocols

🧭 1. Agentic Commerce Protocol (ACP) link

Bootcamp Notes

Setting Up Python Environment link

Glossary

General Links and Resources link

– A – link

– B – link

– C – link

– D – link

– E – link

🧠 3. Summary of Sparse Vector vs. Embedding link

x402 | Conceptual Flow

0️⃣ Links to Check Out link

1️⃣ A2A (Agent-to-Agent) — The Intent Layer link

MCP Servers & Governance

Structured Outputs

Structured Outputs - Notes link

Generating a Table link

Using format strings to build prompts link

Using conditional prompt language link

“Chain of Thought” Prompting link

Assignment #1

Homework #1 link

Prompt Engineering link

main.py link

AI Applications

Planning AI Applications link

Use Case Evaluation link

Assignment #2

Homework #2 link

Brief Notes link

Assignment #3 - Capstone Project Proposal

Homework #3 link

Brief Notes link

Zach AI Agent feedback link

Prompt Engineering

Notes link

Principles of Prompting link

Basic OpenAI API Code link

Neural Networks

Perceptrons link

Intuition link

Advanced Prompt Engineering

“Shot” Prompting link

Large Language Models

Resources and Links link

Notes on Large Language Models link

LLM Model API Behavior link

Embeddings / Vector Representations link

Prompts link

ChatML Format link

Large Language vs. Foundation Models link

Understanding Foundational Models

Training Data link

Modeling link

Model Architecture link

Model Size link

Post Training link

Infererence link

Agentic AI

Links & Resources link

Background & Intro link

Yeh’s Framework (or “Equation”) link

Cook/Kitchen AI Model Analogy

The Full Kitchen Story link

High Level Notes

Three Principles of Semantic Layering

Principle #1 | Data Curation

OpenAI Chat Completion Request

Classification

Zero Shot

Notes

Links & Resources

Adal Prompts

🧭 1. Agentic Commerce Protocol (ACP)

Setting Up Python Environment

General Links and Resources

– A –

– B –

– C –

– D –

– E –

🧠 3. Summary of Sparse Vector vs. Embedding

0️⃣ Links to Check Out

1️⃣ A2A (Agent-to-Agent) — The Intent Layer

Structured Outputs - Notes

Generating a Table

Using format strings to build prompts

Using conditional prompt language

“Chain of Thought” Prompting

Homework #1

Prompt Engineering

`main.py`

Planning AI Applications

Use Case Evaluation

Homework #2

Brief Notes

Homework #3

Brief Notes

Zach AI Agent feedback

Notes

Principles of Prompting

Basic OpenAI API Code

Perceptrons

Intuition

“Shot” Prompting

Resources and Links

Notes on Large Language Models

LLM Model API Behavior

Embeddings / Vector Representations

Prompts

ChatML Format

Large Language vs. Foundation Models

Training Data

Modeling

Model Architecture

Model Size

Post Training

Infererence

Links & Resources

Background & Intro

Yeh’s Framework (or “Equation”)

The Full Kitchen Story

Training Phase = Apprenticeship in the Kitchen

Inference Phase = Restaurant Service