CUDA Setup

CUDA Toolkit

Required: CUDA Toolkit 11.0-12.5

Download from NVIDIA Developer

Nsight Compute

Bundled: Included with CUDA Toolkit

Provides authentic hardware metrics and profiling capabilities.

CUDA Environment Detection

• CUDA Toolkit installation path and version
• nvcc compiler availability and configuration
• NCU (nv-nsight-cu-cli) installation and version
• GPU compute capability and specifications

Hardware Detection

• NVIDIA GPU models and specifications (RTX 40/30/20, GTX 16/10 series)
• GPU memory capacity and bandwidth
• SM (Streaming Multiprocessor) count
• Architecture type (Turing, Ampere, Ada Lovelace, Hopper)

CUDA Path Setup

If automatic detection fails:

1. Go to Settings → CUDA Configuration
2. Set CUDA Toolkit path manually:
   • Windows: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.x
   • Linux: /usr/local/cuda or /opt/cuda
3. Verify NCU installation at: [CUDA_PATH]/nsight-compute-2024.x.x/
4. Test profiling with a simple kernel

Windows UAC/Permissions

Automatic UAC Handling:

• RightNow AI handles UAC elevation automatically for performance counter access
• Automatic UAC prompt when profiling requires elevated permissions
• Registry fixes for Windows performance counters
• No manual setup required

Manual Registry Fix (if needed):

# Run as Administrator
reg add "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\PerfProc\Performance" /v "Disable Performance Counters" /t REG_DWORD /d 0 /f

Linux Permissions & Setup

Required Permissions:

# Add user to GPU access groups
sudo usermod -a -G video $USER
sudo usermod -a -G render $USER

# For NCU profiling (some distributions)
sudo usermod -a -G perf_users $USER

# Reboot after group changes
sudo reboot

GPU Metrics Access:

# Enable GPU performance counters (NVIDIA driver 470+)
sudo nvidia-smi -pm 1
sudo nvidia-smi -lgc 0,1980

# Verify nvidia-smi access
nvidia-smi

Automatic Multi-GPU Detection

• RightNow AI detects all NVIDIA GPUs in the system
• GPU selection dropdown in profiling interface
• Cross-GPU performance comparison

Device Selection

1. Open profiling panel
2. Select target GPU from device dropdown
3. Profile kernels with cudaSetDevice() targeting specific GPU
4. Compare performance across different GPU architectures

Supported Configurations

• Single GPU: RTX 4090, RTX 4080, RTX 3090, etc.
• Multi-GPU: SLI/NVLink configurations
• Mixed Architectures: Different GPU generations in same system

NCU Not Found

1. Ensure CUDA Toolkit is installed (not just runtime)
2. Check NCU binary exists: nv-nsight-cu-cli --version
3. Add CUDA bin directory to PATH

Permission Denied

1. Windows: Run RightNow AI as Administrator or allow UAC prompt
2. Linux: Ensure user is in correct groups (video, render, perf_users)
3. Check GPU driver supports profiling (R470+ recommended)

Profiling Fails

1. Verify GPU supports compute capability 3.5+
2. Close other GPU-intensive applications
3. Check CUDA application compiles and runs normally
4. Try profiling with simple kernel first