Ab­stract

Gi­gan­tic rates of data pro­duc­tion in the era of Big Data, In­ter­net of Thing (IoT), and Smart Cyber Phys­i­cal Sys­tems (CPS) pose in­ces­santly es­ca­lat­ing de­mands for mas­sive data pro­cess­ing, stor­age, and trans­mis­sion while con­tin­u­ously in­ter­act­ing with the phys­i­cal world using edge sen­sors and ac­tu­a­tors. For IoT sys­tems, there is now a strong trend to move the in­tel­li­gence from the cloud to the edge or the ex­treme edge (known as TinyML). Yet, this shift to edge AI sys­tems re­quires to de­sign pow­er­ful ma­chine learn­ing sys­tems under very strict re­source con­straints. This poses a dif­fi­cult de­sign task that needs to take the com­plete sys­tem stack from ma­chine learn­ing al­go­rithm, to model op­ti­miza­tion and com­pres­sion, to soft­ware im­ple­men­ta­tion, to hard­ware plat­form and ML ac­cel­er­a­tor de­sign into ac­count. This paper dis­cusses the open re­search chal­lenges to achieve such a holis­tic De­sign Space Ex­plo­ration for a HW/SW Co-de­sign for Edge AI Sys­tems and dis­cusses the cur­rent state with three cur­rently de­vel­oped flows: one de­sign flow for sys­tems with tightly-cou­pled ac­cel­er­a­tor ar­chi­tec­tures based on RISC-V, one ap­proach using loosely-cou­pled, ap­pli­ca­tion-spe­cific ac­cel­er­a­tors as well as one frame­work that in­te­grates soft­ware and hard­ware op­ti­miza­tion tech­niques to built ef­fi­cient Deep Neural Net­work (DNN) sys­tems.