Sequential decision-making under uncertainty requires balancing exploration and exploitation. The multi-armed bandit framework formalizes this dilemma, while contextual bandits extend it to incorporate side information. In this work, we analyze the upper confidence bound (UCB) and linear UCB (LinUCB) algorithms, which navigate this trade-off using the principle of optimism in the face of uncertainty. We provide a rigorous theoretical exposition, rederiving finite-time bounds that establish sublinear regret guarantees for both algorithms. To complement our theoretical findings, we conduct a comparative empirical evaluation using synthetic data and the MovieLens 1M dataset. Our results highlight the severe impact of model misspecification and the curse of dimensionality.
- Seminar Paper | Graded: 1.0
This thesis investigates the Budgeted Contextual Multi-Armed Bandit (BCMAB) problem, where a decision-maker selects one of N arms in each round t, receiving both a stochastic reward and an associated cost. The reward and cost distributions depend on contextual information xt, which is revealed at the beginning of each round. The objective is to maximize rewards while adhering to a predefined budget, requiring an efficient trade-off between cost and reward in the decision-making process. While existing research primarily addresses either contextual bandit algorithms or budgeted bandit frameworks in isolation, their combination remains underexplored. In this thesis, we propose novel algorithms that integrate both aspects by extending established approaches from Contextual and Budgeted Multi-Armed Bandits. These algorithms are systematically evaluated across diverse environments, focusing on cumulative regret and variance as the primary performance metrics. Furthermore, we analyze their behavior under varying conditions, provide empirical insights and hypotheses into their performance, and outline potential strategies for mitigating their limitations.
- Bachelor Thesis | Graded: 1.0
This was my first seminar paper. It is placed here for reference to track my progress. Abstract: In today’s digital era, we face the significant challenge of developing technologies that are not only effective and resource-efficient, but also sustainable and future-proof. However, we frequently encounter applications and architectures that fail to meet the demands of sustainability and the needs of future generations. This issue raises questions that go far beyond the mere technological advancement of software, touching on ecological, social, individual, and economical aspects. This seminar paper aims to explore the key factors contributing to the lack of sustainability in current technological solutions and introduces solution approaches found in literature, from an architectural perspective. Additionally, it provides an introduction to Green IT, explaining its focus points and presenting approaches to increasing sustainability in IT. Through this exploration, the paper seeks to highlight the importance of sustainable practices in technology development and offer practical strategies for achieving this goal from the literature
- Seminar Paper | Graded: 1.7