INTRODUCTION

The PKU-Vidar-DVS dataset is a large-scale multimodal neuromorphic object detection dataset with temporally continuous labels. It is constructed by the National Engineering Research Center for Visual Technology, Peking University.

Collection Steps and Calibration. This dataset is recorded using our hybrid camera system, which includes a Vidar (resolution 400*250) and a DAVIS346. As shown in Fig.1, the input light is equally divided into Vidar and DAVIS346 via a beam splitter (i.e., Thorlabs CCM1-BS013). On this basis, we design the spatio-temporal calibration procedures to synchronize two cameras within the shared view at the same time.

Data recordings and Annotation. Our PKU-Vidar-DVS dataset contains 9 indoor and outdoor challenging scenarios (see Fig. 2) by considering velocity distribution, illumination change, category diversity, and object scale, etc. We use the hybrid camera system to record 490 sequences including Vidar spikes and DVS events. In each sequence, we collect approximately 5 seconds as the raw data pool. To provide bounding boxes from asynchronous visual streams, frames are reconstructed from Vidar spikes at 50 FPS. After spatio-temporal calibration, all labels are provided by a well-trained professional annotation team.

Data Statistics. Manual annotations in the recordings are provided at a frequency of 50 Hz. As a result, this dataset has 103.3k labeled timestamps and 229.5k labels in total. Afterward, we split them into three subsets for training, validation, and testing. Notably, this is the first work to build a neuromorphic multimodal object detection dataset involving high-speed and low-light scenarios. Besides, more details can be found in Table 1.

Table 1 The details of the PKU-Vidar-DVS dataset.

LICENCE

1. Vidar spikes, DVS events, and the corresponding annotation results can only be used for ACADEMIC PURPOSES. No COMERCIAL USE is allowed.
2. Copyright @ National Engineering Research Center for Visual Technology and Institute of Digital Media, Peking University (PKU-IDM). All rights reserved.

All publications using the PKU-Vidar-DVS dataset should cite the paper below:

Jianing Li, Xiao Wang, Lin Zhu, Jia Li, Tiejun Huang, Yonghong Tian. Retinomorphic object detection in asynchronous visual streams. Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2022.

DOWNLOAD

You can download directly from here .

Email: pkuml at pku.edu.cn

Address: Room 2604, Science Building No.2, Peking University, No.5 Yiheyuan Road, Haidian District, Beijing, P.R.China.

Networks of Spiking Neurons: The Third Generation of Neural Network Models

A review of learning in biologically plausible spiking neural networks

Rethinking the performance comparison between SNNS and ANNS

Long Short-Term Memory Spiking Networks and Their Applications

Towards spike-based machine intelligence with neuromorphic computing

Event-driven Random Backpropagation: Enabling Neuromorphic Deep Learning Machines

S4NN: temporal backpropagation for spiking neural networks with one spike per neuron

Visualizing a joint future of neuroscience and neuromorphic engineering

Surrogate Gradient Learning in Spiking Neural Networks

Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks

SLAYER: Spike Layer Error Reassignment in Time

Inherent Adversarial Robustness of Deep Spiking Neural Networks: Effects of Discrete Input Encoding and Non-Linear Activations

Exploring Adversarial Attack in Spiking Neural Networks with Spike-Compatible Gradient

The PKU-AIR300 Dataset is a new large-scale challenging aircraft dataset. It contains 320,000 annotated color images from 300 different classes in total. Each category contains 100 images at least, and a maximum of 10,000 images, which leads to the long tail distribution. According to the number of images in each category, it is divided all classes into two parts with 180 known classes for training and 120 novel unknown classes for testing respectively.

LICENSE

• The images the corresponding annotation results can only be used for ACADEMIC PURPOSES. NO COMERCIAL USE is allowed.
• Copyright National Engineering Laboratory for Video Technology (NELVT) and Institute of Digital Media, Peking University. All rights reserved.

All publications using Air-300 Dataset should cite the paper below:

Guangyao Chen, Limeng Qiao, Yemin Shi, Peixi Peng, Jia Li, Tiejun Huang, Shiliang Pu and Yonghong Tian. Learning Open Set Network with Discriminative Reciprocal Points. ECCV 2020.

DOWNLOAD

• You can download the agreement (pdf) by clicking the DOWNLOAD link.
• Contact E-mail: pkuml at pku.edu.cn and mail the scanned vision.

The PKU-Masked-Face Dataset is constructed by National Engineering Laboratory for Video Technology (NELVT), Peking University.

The dataset contains 10,301 face images of 1,018 identities. Each identity has masked and common face images with various orientations, lighting conditions and mask types. Most identities have 5 holistic face images and 5 masked face images with 5 different views: front, left, right, up and down.

LICENSE

• The videos and the corresponding annotation results can only be used for ACADEMIC PURPOSES. NO COMERCIAL USE is allowed.
• Copyright © National Engineering Laboratory for Video Technology (NELVT) and Institute of Digital Media, Peking University (PKU-IDM). All rights reserved.

All publications using PKU-Masked-Face Dataset should cite the paper below:

• Feifei Ding, Peixi Peng, Yangru Huang, Mengyue Geng and Yonghong Tian. Masked Face Recognition with Latent Part Detection. ACM Multimedia 2020.

Note: The dataset is different from MFI and MFV proposed in the paper. The facial photos in MFI and MFV have potential illegal risks of privacy leakage. The PKU-Masked-Face Dataset is larger and harder than MFI and MFV.

The experimental results of our model on this dataset are shown in the following table. We use the masked face images as the query set and the holistic face images as the gallery set. ResNet50 is used as the baseline model.

DOWNLOAD

You can download the agreement (pdf) from here. Please make sure that a permanent/long-term responsible person (e.g., professor, PI) fills in the agreement with a handwriting signature.  After filling it, please send the electrical version to our Email: pkuml at pku.edu.cn (Subject: PKU-Masked-Face-Agreement) .

Please send it through an academic or institute email-addresses such as xxx at xxx.edu.xx. Requests from free email addresses (outlook, gmail, qq etc) will be kindly refused.

After confirming your information, we will send the download link and password to you via Email. You need to follow the agreement.

Usually we will reply in a week. But sometimes the mail does not arrive and display successfully for some unknown reason. If this happened, please change the content or title and try sending again.

The PKU-Spike-Recon dataset is constructed by National Engineering Laboratory for Video Technology (NELVT), Peking University. The goals to create the PKU-Spike-Recon dataset include:

1. providing the worldwide researchers of the neuromorphic vision community a spike dataset for evaluating their algorithms;
2. facilitating the development of reconstruction technologies by providing several spike sequences with different motion speed or at different light conditions.

Therefore, the PKU-Spike-Recon dataset is now partly made available for the academic purpose only on a case-by-case basis. The dataset contains Class A (normal speed) and Class B (high speed), which are recorded by spike camera with 40000 FPS. We also provide a spike player for playback spike sequences and related decoding code. Additionally, more details are illustrated in the following table.

The PKU-Spike-Recon dataset is constructed by National Engineering Laboratory for Video Technology (NELVT), Peking University, sponsored by the National Basic Research Program of China and Chinese National Natural Science Foundation. The NELVT at Peking University is serving as the technical agent for distribution of the dataset and reserves the copyright of all the sequences in the dataset. Any researcher who requests the PKU-Spike-Recon dataset must sign this agreement and thereby agrees to observe the restrictions listed in this document. Failure to observe the restrictions will result in access being denied for the request of the future version of the PKU-Spike-Recon dataset and being subject to civil damages in the case of publication of sequences that have not been approved for release.

LICENSE

•  The spike sequences for download are part of the PKU-Spike-Recon dataset.
•  The sequences can only be used for ACADEMIC PURPOSES. NO COMERCIAL USE is allowed.
•  Copyright © National Engineering Laboratory for Video Technology (NELVT) and Institute of Digital Media, Peking University (PKU-IDM). All rights reserved.

All publications using PKU-Spike-Recon should cite the papers below:

1. L. Zhu, S. Dong, J. Li, T. Huang, Y. Tian, Retina-like Visual Image Reconstruction via Spiking Neural Model, CVPR 2020
2. L. Zhu, S. Dong, T. Huang, Y. Tian, A retina-inspired sampling method for visual texture reconstruction, ICME 2019

Download(The dataset is available soon)

You can download the agreement(pdf) by clicking the DOWNLOAD link .

After filling it, please send the electrical version to our Email: pkuml at pku.edu.cn (Subject: PKU-Spike-Recon Agreement)

The PKU-Spike-High-Speed Dataset is constructed by National Engineering Laboratory for Video Technology (NELVT), Peking University.

The goals to create this dataset includes: (1) reconstructing high-speed moving target; (2) Capturing natural scenes for high-speed moving camera. Therefore, the first available part contains Class A (moving target) and Class B (moving camera) with eight spike sequences, which are recorded by a retina-based camera, namely fovea-like sampling model (FSM). In particular, our FSM has high temporal resolution (40k FPS) and can reconstruct texture image with 400*250 pixel. We also provide two spike players (jspikeplayer.jar and SpikePlayer.exe) for playback spike sequences. Additionally, more details are illustrated in the following table.

Sequence		Scene reconstruction	Length (s)	Spike numbers
Class A moving target	car-100km/h		0.2	102206031
	bus		0.42	211084678
	ratation1-2600r/min		2	396742501
	ratation2-2600r/min		2	407620564
Class B moving camera	train-350km/h		0.2	42898223
	forest		0.22	93319068
	viaduct bridge		0.22	136859111
	railway		0.22	87866720

Note: Class A is captured by the fixed FSM, and class B is recorded by the FSM in high railway with the speed of 350 km/h.

The PKU-Spike-High-Speed Dataset is now partly made available for the academic purpose only on a case-by-case basis. More sequences are under evaluation which will be public in the future.

LICENSE

• The sequences can only be used for ACADEMIC PURPOSES. NO COMERCIAL USE is allowed.
• Copyright National Engineering Laboratory for Video Technology (NELVT) and Institute of Digital Media, Peking University.

All publications using PKU-Spike-High-Speed Dataset should cite the papers below:

Lin Zhu, Siwei Dong, Tiejun Huang, Yonghong Tian. A retina-inspired sampling method for visual texture reconstruction, IEEE International Conference on Multimedia and Expo (ICME), 2019.

DOWNLOAD

• You can download the agreement (pdf) by clicking the DOWNLOAD link.
• Contact E-mail: pkuml at pku.edu.cn

The PKU-Spike-A dataset is constructed by National Engineering Laboratory for Video Technology (NELVT), Peking University, sponsored by the National Basic Research Program of China and Chinese National Natural Science Foundation. The goals to create the PKU-Spike-A include:
(1) providing the worldwide researchers of the spike coding community a large-scale spike dataset for evaluating their coding algorithms;
(2) facilitating the development of high-efficient spike coding technologies by providing large-scale spike sequences with different depth of field or at different luminance and time conditions.
Therefore, the PKU-Spike-A dataset is now partly made available for the academic purpose only on a case-by-case basis. The first available part contains Class A (normal speed) and Class B (high speed) with six spike sequences in total. More sequences are under evaluation which will be public available in the near future.

The PKU-Spike-A dataset is constructed by National Engineering Laboratory for Video Technology (NELVT), Peking University, sponsored by the National Basic Research Program of China and Chinese National Natural Science Foundation. The NELVT at Peking University is serving as the technical agent for distribution of the dataset and reserves the copyright of all the sequences in the dataset. Any researcher who requests the PKU-Spike-A dataset must sign this agreement and thereby agrees to observe the restrictions listed in this document. Failure to observe the restrictions will result in access being denied for the request of the future version of the PKU-Spike-A dataset and being subject to civil damages in the case of publication of sequences that have not been approved for release.

LICENSE

• The spike sequences for download are part of the PKU-Spike-A dataset.
• The sequences can only be used for ACADEMIC PURPOSES. NO COMERCIAL USE is allowed.
• Copyright © National Engineering Laboratory for Video Technology (NELVT) and Institute of Digital Media, Peking University (PKU-IDM). All rights reserved.

Download

• You can download the agreement(pdf) by clicking the DOWNLOAD link (available soon).
• After filling it, please send the electrical version to our Email: pkuml at pku.edu.cn (Subject: PKU-Spike-A Agreement) and mail the paper version to our lab: Room 2604, Science Building No. 2, Peking University, No.5 Yiheyuan Road, Haidian District, Beijing, P.R.China .

INTRODUCTION

The PKU-DDD17-CAR Dataset is constructed by National Engineering Laboratory for Video Technology (NELVT), Peking University.

The dataset contains 3155 hybrid sequences in driving scenes, which consist of images, event streams and handed car labels. Specially, we first collect from DDD17 dataset, which has over 400GB and 12 hours of 346*260 pixel DAVIS sensor recording highway and city driving in daytime and night-fall conditions. Then, we provide the hand-label dataset by synchronizing frames and event streams. As shown in Figure 1, four representative scenes are motion blur, overexposure, low-night and normal-light. In addition, more details are illustrated in Table 1.

Figure 1: Four representative scenes in driving scenes.

LICENCE

All publications using PKU-DDD17-CAR dataset should cite the papers below:

• Jianing Li, Siwei Dong, Zhaofei Yu, Yonghong Tian, Tiejun Huang. Event-based vision Enhanced: A Joint Detection Framework in Autonomous Driving, IEEE International Conference on Multimedia and Expo (ICME), 2019.
• Janathan Blnas, Danlel Nill, Shih-Chil Liu, Tobi Delbruck. DDD17: End-To-End DAVIS Driving Dataset. Proceedings of 34th International Conference on Machine Learning (ICML), 2017.

DOWNLOAD

You can download directly from here. password:ddd17-201904

Email: pkuml at pku.edu.cn

Address: Room 2604, Science Building No.2, Peking University, No.5 Yiheyuan Road, Haidian District, Beijing, P.R.China.

INTRODUCTION

The PKU Sketch Re-ID dataset is constructed by National Engineering Laboratory for Video Technology (NELVT), Peking University.

The dataset contains 200 persons, each of which has one sketch and two photos. Photos of each person were captured during daytime by two cross-view cameras. We cropped the raw images (or video frames) manually to make sure that every photo contains the one specific person. We have a total of 5 artists to draw all persons’ sketches and every artist has his own painting style.

LICENSE 

• The images and the corresponding annotation results can only be used for ACADEMIC PURPOSES. NO COMERCIAL USE is allowed.
• Copyright © National Engineering Laboratory for Video Technology (NELVT) and Institute of Digital Media, Peking University (PKU-IDM). All rights reserved.

All publications using PKU SketchRe-ID dataset should cite the paper below:

Lu Pang, Yaowei Wang, Yi-Zhe Song, Tiejun Huang, Yonghong Tian; Cross-Domain Adversarial Feature Learning for Sketch Re-identification; ACM Multimedia 2018

DOWNLOAD

You can download the agreement (pdf) from here. After filling it, please send the electrical version to our Email: pkuml at pku.edu.cn (Subject: PKU-SketchReID-Agreement) 

Please send it through an academic or institute email-addresses such as xxx at xxx.edu.xx. Requests from free email addresses (outlook, gmail, qq etc) will be kindly refused.

After confirming your information, we will send the download link and password to you via Email. You need to follow the agreement.

Usually we will reply in a week. But sometimes the mail does not arrive and display successfully for some unknown reason. If this happened, please change the content or title and try sending again.

The PKU-DVS dataset is constructed by National Engineering Laboratory for Video Technology (NELVT), Peking University, sponsored by the National Basic Research Program of China and Chinese National Natural Science Foundation. The goals to create the PKU-DVS include:
(1) providing the worldwide researchers of the spike coding community a large-scale DVS spike dataset for evaluating their coding algorithms;
(2) facilitating the development of high-efficient spike coding technologies by providing large-scale spike sequences with different depth of field or at different luminance and time conditions.
Therefore, the PKU-DVS dataset is now partly made available for the academic purpose only on a case-by-case basis. The first available part contains Class A (indoor) and Class B (outdoor) with seven and six spike sequences, respectively. More sequences are under evaluation which will be public available in the near future.

The PKU-DVS dataset is constructed by National Engineering Laboratory for Video Technology (NELVT), Peking University, sponsored by the National Basic Research Program of China and Chinese National Natural Science Foundation. The NELVT at Peking University is serving as the technical agent for distribution of the dataset and reserves the copyright of all the sequences in the dataset. Any researcher who requests the PKU-DVS dataset must sign this agreement and thereby agrees to observe the restrictions listed in this document. Failure to observe the restrictions will result in access being denied for the request of the future version of the PKU-DVS dataset and being subject to civil damages in the case of publication of sequences that have not been approved for release.