Syllabus

Course Information¶

- **Semester:** Fall 2025  
- **Course Prefix/Number:** COT6905  
- **Course Title:** Directed Study: Advanced Optimization Algorithms for Large Foundational AI Models  
- **Course Credit Hours:** 3.0  
- **Class Meeting Time:** Weekly, by appointment  
- **Instructor:** Prof. Brian Jalaian  
  - Computer Science  
  - Building 4, Room 434 (Main Campus)  
  - Email: bjalaian@uwf.edu

Course Description¶

This directed study investigates the advanced mathematical and optimization foundations required for training and deploying large-scale foundational AI models. Topics include:

• Matrix theory for understanding and improving optimization algorithms (vector spaces, ranks, block operations, decompositions)
• Tensor factorization techniques relevant to modern architectures
• Stochastic gradient descent (SGD) and adaptive optimizers
• Stability of large-scale training (conditioning, vanishing/exploding gradients)
• Second-order optimization methods (natural gradient, quasi-Newton)
• Model compression strategies: quantization, pruning, distillation, low-rank approximations
• Scaling laws, training dynamics, and hardware-aware optimization
• Evaluation metrics beyond accuracy: FLOPs, latency, memory footprint, energy cost

Students will develop the mathematical and computational background to contribute to state-of-the-art research in model compression and efficient large-scale AI.

Topics Covered¶

• Matrix theory foundations for deep learning
• Block matrix multiplications and tensor factorization
• Stochastic gradient descent and adaptive optimizers (Adam, RMSProp, etc.)
• Numerical stability in optimization: conditioning, exploding/vanishing gradients, preconditioning
• Second-order optimization algorithms: natural gradient, quasi-Newton, Fisher-based methods
• Probabilistic and information-theoretic tools: entropy, KL divergence, variational inference
• Model compression: quantization, structured/unstructured pruning, knowledge distillation
• Hardware-aware optimization: mixed-precision training, quantization-aware training
• Scaling laws and training dynamics of large AI models
• Evaluation metrics for efficiency: FLOPs, throughput, latency, memory footprint, and accuracy trade-offs

Course Work¶

Students are expected to create an online e-Book synthesizing the material, with simplified explanations, references to videos, and interactive examples. Computational notebooks should be organized in a public GitHub repository for reproducibility.

Expected Outcomes¶

After this course, students will be able to:

• Demonstrate mastery of matrix theory applied to deep learning
• Explain modern model compression techniques from a matrix theory perspective
• Understand advanced optimization algorithms, including second-order methods
• Apply probabilistic and information-theoretic concepts to optimization and compression
• Produce professional-quality documentation and computational notebooks

Grading¶

• 100% – Final e-Book report and GitHub deliverables

Exams¶

• There are no exams; grades are based solely on reports and supplemental deliverables.

Academic Conduct¶

Students are expected to comply with the Student Code of Academic Conduct regarding plagiarism and misconduct. More information: Dean of Students – Academic Conduct

Minimum Technical Skills¶

Students should be able to:

• Activate a MyUWF student account
• Access MyUWF portal 2–3 times per week
• Access UWF email 2–3 times per week
• Perform basic word processing

Student use of technology is governed by the Computing Resources Usage Agreement and the Student Communications Policy.

Course Modality¶

Faculty may adjust the modality of class meetings due to weather, pandemics, or other events. Flexibility is required to maintain continuity.

TurnItIn¶

UWF maintains a license for Turnitin to check originality. Instructors may also use other methods as needed.

AI Usage Policy¶

Generative AI tools are permitted for:

• Brainstorming ideas
• Finding information
• Drafting outlines
• Developing code snippets
• Grammar/style checks

Usage must be documented and cited. Unauthorized use may result in a zero.

Discrimination, Harassment, and Civil Discourse¶

• Title IX compliance is required. Support services are available confidentially.
• Civil discourse is expected. Understanding ideas does not require agreement.

Health, Safety, and Support Services¶

• Student Health Clinic: 850-474-2172
• Counseling & Psychological Services: 850-474-2420
• TAO Online Self-Help Program: Available 24/7
• TogetherAll Peer Support: http://uwf.edu/togetherall
• ArgoWell Wellness Initiative: http://uwf.edu/argowell
• Ask-a-Librarian Live Chat: Available 8am–11pm Mon–Thu; 8am–4pm Fri; 9am–4pm Sat; 9am–11pm Sun

Resources¶

1. Cybersecurity library resources
2. Writing Lab: Graduate/undergraduate assistants available for review
3. Canvas Support Hotline: 1-844-866-3349
   UWF ITS Help Desk: 850-474-2075, itshelpdesk@uwf.edu
4. Career Development & Community Engagement (CDCE): Resume, cover letter, interview support

Emergency Information¶

• University closures and alerts: UWF Emergency Info
• Mobile Alert and WUWF-FM 88.1 MHz provide official updates
• Hurricane preparation procedures: Emergency Procedures Guide

Other Course Policies¶

• Online Resources: Use Discord for course discussions
• Communication: Contact instructor by appointment; check UWF email regularly
• Class Attendance: Meetings held face-to-face or online
• Withdraw Policy: Check UWF Academic Calendar; late withdrawals = WF
• Incomplete Grades: Only with extenuating circumstances and ≥70% completed work

Note: Any syllabus changes during the semester take precedence.