This app is designed as a learning aid for anyone seeking to become familiar with Generative AI
apps and agents. It's based on the user interface in Microsoft Foundry portal, but does not use
any Azure cloud services.
The app uses AI language models for both GPU and CPU modes. For GPU mode, it uses Microsoft's
Phi-3.5-mini on WebLLM (requires WebGPU API and GPU). For CPU mode, it uses
Phi-2 on wllama (works on any device). If WebLLM fails to load,
the app automatically falls back to CPU mode. You can switch between modes using the Model
dropdown.
The app also makes use of the MobileNetV3 model and the TensorFlow.js framework to implement
image classification.
If GPU and CPU model responses are too slow or become unresponsive on older or lower-spec
computers, switch the Model dropdown to None as a last-resort option. This
mode skips local language model inference and uses a lightweight fallback experience.
Known issues
- The initial download of the Microsoft Phi model may take a few minutes - particularly on
low-bandwidth connections. Subsequent downloads should be quicker.
- Some GPU-enabled computers (particularly those with ARM-based processors) do not support
WebGPU without enabling the Unsafe WebGPU Support browser flag. If your
browser fails to load the Microsoft Phi model, you can try enabling this flag at edge://flags on Microsoft Edge or chrome://flags
on Google Chrome. Disable it again when finished!
- CPU mode (Phi-2) response times are slower than GPU mode (Phi-3-mini). Phi-2 is a larger
model running through WebAssembly, so CPU responses can take noticeably longer. Use the
Model dropdown to select your preferred mode.
- Image analysis is based on MobileNetV3, which is a general-purpose image classification
model. It may not accurately identify all images or provide detailed descriptions.
- Voice mode depends on browser-support for speech recognition. A
fallback offline speech-to-text model will be used in the event of a speech recognition
failure.
- Voice mode performance may vary based on the device's microphone quality and ambient noise
levels. For best results, use a good quality microphone in a quiet environment.