NexaAI VLM Function Call Demo with Google Calendar MCP

Demonstrates function calling capabilities of NexaAI/OmniNeural-4B model, integrated with Google Calendar via MCP protocol.

Features

• Function calling with NexaAI VLM model
• Google Calendar integration via MCP
• Automatic function call parsing and execution
• Multi-modal input support (text, image, audio)
• Web UI and command-line interfaces

Prerequisites

• Python 3 (arm64 architecture recommended)
  • See bindings/python/notebook/windows(arm64).ipynb for setup
• Node.js and npm (for MCP server)
  powershellCopy

  winget install OpenJS.NodeJS.LTS
  

  Restart terminal/IDE after installation.

Installation

pip install -r requirements.txt

Google Calendar Setup

1. Go to Google Cloud Console

2. Create/select a project and enable Google Calendar API

3. Go to OAuth consent screen follow the instructions to configure your consent screen.

4. Create OAuth 2.0 credentials:

   • Go to Credentials
   • Create "Create Credentials" > "OAuth client ID" > Select "Desktop app" > click "Create"
   • Click "Download JSON" and save as gcp-oauth.keys.json on the same directory as this README.md

5. Add your email as a test user in Audience

   • Click "Add User" > enter your email address > click "Save"
   • Note: Test mode tokens expire after 1 week

6. Authenication (only need to do once)

$env:GOOGLE_OAUTH_CREDENTIALS="gcp-oauth.keys.json"
npx @cocal/google-calendar-mcp auth

follow the instructions to authorize the application to access your Google Calendar.

Tip: Ensure the OAuth client ID is enabled for Calendar API at Credentials For detailed setup, see: https://github.com/nspady/google-calendar-mcp?tab=readme-ov-file#google-cloud-setup

Usage

Command Line

# Text only
python main.py --text "what is the time now?"

# Image with text
python main.py --image image.png --text "help me add this event to my calendar"

# Audio with text
python main.py --audio audio.mp3 --text "transcribe and add to calendar"

Web UI

python .\app\flask_ui.py