Ab­stract

Ro­bust per­cep­tion of the en­vi­ron­ment under a va­ri­ety of am­bi­ent con­di­tions is cru­cial for au­tonomous dri­ving. Con­vo­lu­tional Neural Net­works (CNNs) achieve high ac­cu­racy for vi­sion-based ob­ject de­tec­tion, but are strongly af­fected by ad­verse weather con­di­tions such as rain, snow, and fog, as well as soiled sen­sors. We pro­pose phys­i­cally cor­rect sim­u­la­tions of these con­di­tions for vi­sion-based sys­tems, since pub­licly avail­able data sets lack sce­nar­ios with dif­fer­ent en­vi­ron­men­tal con­di­tions. In ad­di­tion, we pro­vide a data set of real im­ages con­tain­ing ad­verse weather for eval­u­a­tion. By train­ing CNNs with aug­mented data, we achieve a sig­nif­i­cant im­prove­ment in ro­bust­ness for ob­ject de­tec­tion. Fur­ther­more, we pre­sent the ad­van­tages of co­op­er­a­tive per­cep­tion to com­pen­sate for lim­ited sen­sor ranges of local per­cep­tion. A key as­pect of au­tonomous dri­ving is safety; there­fore, a ro­bust­ness eval­u­a­tion of the per­cep­tion sys­tem is nec­es­sary, which re­quires an ap­pro­pri­ate safety met­ric. In con­trast to ex­ist­ing ap­proaches, our safety met­ric fo­cuses on scene se­man­tics and the rel­e­vance of sur­round­ing ob­jects. The per­for­mance of our ap­proaches is eval­u­ated using real-world data as well as aug­mented and vir­tual re­al­ity sce­nar­ios.