We design differentially private algorithms for the problem of online linear optimization in the full information and bandit settings with optimal O(T^0.5) regret bounds. In the full-information setting, our results demonstrate that ε-differential privacy may be ensured for free -- in particular, the regret bounds scale as O(T^0.5+1/ε). For bandit linear optimization, and as a special case, for non-stochastic multi-armed bandits, the proposed algorithm achieves a regret of O(T^0.5/ε), while the previously best known regret bound was O(T^{2/3}/ε).