In this paper, we propose an energy management strategy based on deep reinforcement learning for a hybrid battery system in electric vehicles consisting of a high-energy and a high-power battery pack. The energy management strategy of the hybrid battery system was developed based on the electrical and thermal characterization of the battery cells, aiming at minimizing the energy loss and increasing both the electrical and thermal safety level of the whole system. Primarily, we designed a novel reward term to explore the optimal operating range of the high-power pack without imposing a rigid constraint of state of charge. Furthermore, various load profiles were randomly combined to train the deep Q-learning model, which avoided the overfitting problem. The training and validation results showed both the effectiveness and reliability of the proposed strategy in loss reduction and safety enhancement. The proposed energy management strategy has demonstrated its superiority over the reinforcement learning-based methods in both computation time and energy loss reduction of the hybrid battery system, highlighting the use of such an approach in future energy management systems.

Deep reinforcement learning-based energy management of hybrid battery systems in electric vehicles

The hybrid energy storage system (HESS), which is composed of battery and ultracapacitor, is established to enhance the performance of an electric vehicle (EV). Moreover, several studies highlight the gains of split the traction power demand among electric motors (EMs) with different characteristics and drivetrain configurations, allowing these to operate at higher efficiency. In this paper, we unite the advantages of the multi EM drive train with the HESS power split and propose a novel two HESS system, in which each HESS is in charge of one of the drivetrain configurations (front and rear system). The EV is driven by two in-wheel EM at the front wheels and a single EM assembled with a differential transmission that moves the rear wheels. In this paper, a multiobjective optimization, based on a genetic algorithm (GA), is formulated to minimize the HESS sizing and maximize the driving range of the vehicle. Also employing the fuzzy control, this strategy is responsible to split the power between the front and rear wheels systems in a more reasonable way to satisfy the demands of better performance. The complete strategy has been developed under the FTP-75 (urban), HWFET (highway) and the US06 (high speed and required acceleration) driving cycles using the MATLAB/Simulink software environment. As compared to a similar EV with a single HESS system, the proposed dual-HESS configuration was able to improve the driving range in 145.15 km also decreasing 23.93% of the HESS mass.

Electric Vehicle Powertrain and Fuzzy Control Multi-Objective Optimization, Considering Dual Hybrid Energy Storage Systems

Lithium-ion batteries are widely used in transportation applications due to their outstanding performance in terms of energy and power density as well as efficiency and lifetime. Although various cell chemistries exist, most of today’s electric vehicles on the market have a high-voltage lithium-ion battery system consisting of cells with a graphite-based anode and a metal-oxide cathode. These cells offer a high specific energy density that enables long driving ranges at moderate costs. For applications where power density is the critical design criterion, cells with lithium titanate oxide-based anode materials can be an alternative. These cells offer further advantages such as improved cycle stability and good charge acceptance even at temperatures below 0∘C. This paper presents different applications for high-power batteries in electrified vehicles and compares the requirements for suitable battery cells. After an introduction to lithium titanate oxide as anode material in battery cells, electrical and thermal characteristics are presented. For this reason, measurements were performed with two cells using different cathode active materials and a lithium titanate oxide-based anode. Aging behavior is investigated with lifetime tests performed under high-current cycling conditions at two different ambient temperatures. Differences in the capacity loss rate are shown and lifetime considerations are presented. Furthermore, an incremental capacity analysis is performed at different times in the aging study for a deeper analysis of the aging effects occurring in the two cell types. Finally, cost considerations of lithium titanate oxide-based battery cells with different properties are presented. Varied production volumes are considered and production costs are compared with costs of state-of-the-art graphite-based high-energy battery cells.

Lithium titanate oxide battery cells for high-power automotive applications – Electro-thermal properties, aging behavior and cost considerations

Cloud-based health-conscious energy management of hybrid battery systems in electric vehicles with deep reinforcement learning

Improving inverter selection for electric vehicles is a must when tackling overall vehicle efficiency and reduction of traction system losses. This paper investigates the efficiency benefits achieved by using an 800V dc bus voltage and wideband gap SiC carbide devices rather than a conventional IGBT inverter and a 400V dc bus. Inverters utilizing 600V IGBTs, 1200V IGBTs, 1200V hybrid silicon carbide (SiC) devices, and 1200V full SiC devices are modeled. The modeled loss for each inverter type is incorporated into a full electro-mechanical vehicle model as a way of accurately representing their behavior in an application. Measured data from experimental testing of a 180kW traction machine and a 1200V IGBT inverter were also used in developing and verifying the validity of the inverter model. Simulation results show around a 5% reduction in an electric vehicle’s energy consumption, and consequently a 5% increase in range, when a full SiC inverter is used. All results are normalized to a 600V IGBT inverter, showing that all the 1200V devices are more efficient and that the SiC devices show the most improvement overall.

Comparison of IGBT and SiC Inverter Loss for 400V and 800V DC Bus Electric Vehicle Drivetrains

A semi-active hybrid energy storage system, consisting of a Li-ion battery pack, dc/dc converter, and Li-ion capacitor pack is developed for a range extended plug-in vehicle application. The vehicle has a series-parallel drivetrain with two electric motors, a gas engine, gearbox, and a clutch to allow the engine to run decoupled from the gearbox in range extending mode. The peak dc electrical requirement of the electric drivetrain is about 175kW, which is similar to the peak power capability of the developed hybrid energy storage system. A model of the prototype hybrid energy storage system, which has the Li-ion capacitor pack connected directly to the motor drive's dc bus and the battery pack connected to the Li-ion capacitor pack via a dc/dc converter, is developed and used to determine the optimal power split between the battery and Li-ion capacitor packs. A dynamic programming algorithm is used to determine the optimal power split, with the optimization goals of reducing energy storage system loss, maximizing regenerative braking energy capture, and minimizing motoring power limiting. The hybridized system is shown to reduce battery pack losses and increase vehicle range compared to a system only utilizing the battery pack.

Optimal performance of a full scale li-ion battery and li-ion capacitor hybrid energy storage system for a plug-in hybrid vehicle

A semi-active hybrid energy storage system, consisting of a Li-ion battery pack, dc/dc converter, and Li-ion capacitor pack was developed for a range extended plug-in vehicle. The vehicle has a series-parallel drivetrain with two electric motors, a gas engine, gearbox, and a clutch to allow the engine to run decoupled from the gearbox in range extending mode. The peak dc electrical requirement of the electric drivetrain is about 175 kW, which is similar to the peak power capability of the developed hybrid energy storage system. A model of the prototype hybrid energy storage system, which has the Li-ion capacitor pack connected directly to the motor drive's dc bus and the battery pack connected to the Li-ion capacitor pack via a dc/dc converter, is developed and used to determine the optimal power split between the battery and Li-ion capacitor packs and for tuning the developed real-time control system. The real-time control system is shown through modeling and experimental testing of the full scale hybrid energy storage systems to reduce battery pack losses, increase vehicle range, and to have performance approaching that of the optimal control solution as calculated via dynamic programming.

Megan Wood

Papers

Optimal performance of a full scale li-ion battery and li-ion capacitor hybrid energy storage system for a plug-in hybrid vehicle

Real-Time Control of a Full Scale Li-ion Battery and Li-ion Capacitor Hybrid Energy Storage System for a Plug-in Hybrid Vehicle