Embedded Systems

Automated HW/SW Co-Design for Edge AI: State, Challenges and Steps Ahead

by Oliver Bring­mann, Wolf­gang Ecker, Ingo Feld­ner, Adrian Frischknecht, Christoph Gerum, Timo Hämäläinen, Muham­mad Ab­dul­lah Hanif, Michael J. Klaiber, Daniel Mueller-Gritschneder, Paul Palom­ero Bernardo, Se­bas­t­ian Pre­beck, and Muham­mad Shafique
In Pro­ceed­ings of the 2021 In­ter­na­tional Con­fer­ence on Hard­ware/Soft­ware Code­sign and Sys­tem Syn­the­sis, pages 11–20. As­so­ci­a­tion for Com­put­ing Ma­chin­ery, 2021.

Ab­stract

Gi­gan­tic rates of data pro­duc­tion in the era of Big Data, In­ter­net of Thing (IoT), and Smart Cyber Phys­i­cal Sys­tems (CPS) pose in­ces­santly es­ca­lat­ing de­mands for mas­sive data pro­cess­ing, stor­age, and trans­mis­sion while con­tin­u­ously in­ter­act­ing with the phys­i­cal world using edge sen­sors and ac­tu­a­tors. For IoT sys­tems, there is now a strong trend to move the in­tel­li­gence from the cloud to the edge or the ex­treme edge (known as TinyML). Yet, this shift to edge AI sys­tems re­quires to de­sign pow­er­ful ma­chine learn­ing sys­tems under very strict re­source con­straints. This poses a dif­fi­cult de­sign task that needs to take the com­plete sys­tem stack from ma­chine learn­ing al­go­rithm, to model op­ti­miza­tion and com­pres­sion, to soft­ware im­ple­men­ta­tion, to hard­ware plat­form and ML ac­cel­er­a­tor de­sign into ac­count. This paper dis­cusses the open re­search chal­lenges to achieve such a holis­tic De­sign Space Ex­plo­ration for a HW/SW Co-de­sign for Edge AI Sys­tems and dis­cusses the cur­rent state with three cur­rently de­vel­oped flows: one de­sign flow for sys­tems with tightly-cou­pled ac­cel­er­a­tor ar­chi­tec­tures based on RISC-V, one ap­proach using loosely-cou­pled, ap­pli­ca­tion-spe­cific ac­cel­er­a­tors as well as one frame­work that in­te­grates soft­ware and hard­ware op­ti­miza­tion tech­niques to built ef­fi­cient Deep Neural Net­work (DNN) sys­tems.