The Technology Of "Pirated Clothing" In Terms Of Biometric Identification And Graph Analysis

The idea behind this is that based on the clothing regional expression retrieval model, the similarity and learning of clothing in the image are studied and measured.

The research results have been included in CVPR 2020 and have been selected as Oral papers.

Ali safety Turing laboratory said that the work will use Ali original protection platform, in the Taobao, Tmall and other Ali business platform online, providing the ability to detect infringement.

Meticulous plagiarism is one foot high, how high is the fake road?

As far as clothing industry is concerned, although counterfeiting has been continuous, piracy plagiarism is still widespread. And from online to offline, plagiarism is becoming more and more tricky, and the difficulty of counterfeiting increases year by year. At present, there are only three categories of plagiarism in the field of clothing.

The first category focuses on image embezzlement. Pirates usually use or modify the commercial maps of genuine brands without authorization, such as adding watermark to their stores, or doing some image processing (inversion, scaling, splicing, etc.).

This kind of infringement plagiarism cost is very low, but it is easy to be locked by the image retrieval system of the platform, and then quickly "governance".

The second category is creative embezzlement. Bad businesses directly copy the original product design and creativity of the original merchants, making the same or imitation money.

The cost of such infringement is slightly higher, but the same item retrieval algorithm based on commodity similarity measure can recall and manage them.

The third category is embezzlement, which is to modify some parts of clothing, such as washing manuscripts, such as changing the design styles of necklines, or the layout of printing on the chest, or even changing the styles of garments.

But as shown below, it is still plagiarizing the style and design elements of the genuine brand clothing (the left side is genuine, the right side is pirated), and even sells it as "star money".

This kind of piracy has the highest cost and is not easily locked by traditional algorithms based on the same commodity. Under normal circumstances, the electronic business platform can only be found through manual audit, the cost of counterfeiting is very high.

Is there a way that the system can automatically lock such plagiarism? This is the latest research direction of Ali safety Turing laboratory.

Previously, they proposed a clothing copyright algorithm to locate plagiarism based on attribute aware fine grained similarity learning method, which was captured by AAAI2020.

Now they have put forward a new idea. Based on the clothing regional expression retrieval model, we can learn and measure the similarity of the clothing in the image, so as to achieve more effective anti-counterfeiting.

　 Search algorithm of "pirated clothing image" to sleeves and collar accurately

The definition of pirated clothing is to plagiarize the original design and style of clothing, and modify it in one or two regions to avoid the existing clothing samples screened by the same clothing retrieval model.

In terms of algorithm design, they put forward a regional attention mechanism guided by clothing key points.

First of all, the key points of clothing are predicted by the key points of clothing, which are the key points in clothing images, such as necklines, cuffs, shoulders, armpits and so on.

The number and distribution of key points for each type of clothing vary somewhat, with about 25-40 points per item. According to these key points, the algorithm can partition multiple areas of clothing pictures, such as collar, sleeve, chest, waist area, etc.

Based on the idea of ROI Pooling, regional integration information is decoupled into multiple regionalized feature expressions to independently learn and measure the similarity of features.

At the same time, the key points of clothing combined with the expression of regionalization can be used as a attention mechanism. In the image retrieval network, the weight of key parts is improved, and the weight of non key parts is reduced to enhance the discriminant power of the model to the key parts.

The key point of clothing estimation is that branch and image retrieval branches use the same HR-Net backbone network, and its multilevel parallel structure keeps multi-resolution features while maintaining high resolution.

In the selection of loss function, the mean value loss function is adopted in the estimation of the key points, and the Triplet loss function is adopted in the retrieval branch. The loss function value is no longer the characteristic triplet difference of the whole picture range, but the cumulative result of the difference of the regional characteristics.

The framework of the method is shown in the following figure: the network can be divided into the clothing critical point estimation branch and the clothing retrieval branch. The retrieval network includes two output forms: the same clothing retrieval and the pirated clothing retrieval.

Based on the analysis of the sample of the platform infringement, the Alibaba researchers found that different categories of clothing are easily different from the pirated area. Therefore, it is not enough to decouple the similarity measurement of the clothing image features. It is also necessary to set the differentiated weights for each area of each garment and carry out weighted regional similarity calculation to recall more pirated clothing samples. Ben.

To this end, they set up a dataset named "Fashion Plagiarism Dataset" based on platform pirated clothing data. The dataset contains query images of each original garment corresponding to gallery's "pirated clothing" images, and the data cover four types of short sleeved T-shirts, long sleeved blouses, jackets and dresses.

In this dataset, Fine Tune training is performed on the pre trained network on the Deepfashion2 dataset, and the Coordinate Ascent algorithm is used to iteratively optimize the weights of different clothing categories in order to reduce the loss function value.

The loss function of "piracy clothing" retrieval training process is also based on the Triplet loss function design. Finally, the pirated retrieval network after training can recall the pirated clothing samples in the green box of Output based on the Input clothing image in the above picture.

　　 What is the effect of counterfeiting? Not lose or even surpass the previous SOTA

In the experimental part of the paper, Alibaba researchers first evaluated the retrieval ability of the pirated clothing image of the algorithm on "Fashion Plagiarism Dataset".

In addition to the methods proposed in the paper, they also set two methods for comparison: one is the traditional retrieval method, which uses the same loss function of backbone network and Triplet, but does not contain regionalization feature learning and expression mechanism; the other is the regionalization feature representation mechanism, but uses non Fine. The regional weights obtained by Tune training are mAP.

From the results of the table, we can see that the method used in the paper has achieved the best results in all garment categories.

In addition to the above "piracy clothing retrieval" evaluation experiment, they also conducted clothing key point estimation and the same clothing image retrieval task experiment on the Deepfashion series dataset.

In the estimation of the key points of clothing, Alibaba researchers evaluated the most complex Deepfashion2 data sets. Compared with the existing Match-RCNN, CPN and Simple-Baseline methods, the clothing critical point estimation model achieved the highest mAP results in each subset.

In the same clothing retrieval experiment, they used FashionNet, Match-RCNN, PCB and other methods as contrast, and conducted experiments on Deepfashion and Deepfashion2 respectively.

Among them, Deepfashion mainly focuses on In-shop retrieval scenarios, while Deepfashion2 is for Consumer-to-shop scenarios. The evaluation indexes were Top-N recall and Top-N accuracy respectively.

As shown in the following figure, Ali's method achieves similar results with the SOTA method on the Deepfashion dataset, and the result on the Deepfashion2 dataset is much better than the existing baseline method.

A total of 5 researchers participated in the study. They were from the Alibaba, Zhejiang Gongshang University and the Alibaba / Zhejiang University frontier technology joint research center.

The first author is Yining Lang, second author Yuan He, and third author Fan Yang from Ali safety Turing laboratory. Xue Hui, director of the safety Turing laboratory in Ali, is the author of the article. Zhejiang Gongshang University Jianfeng Dong is also one of the authors of the paper.

Ali safety Turing laboratory was formally established in 2016. It was formerly an Ali security foundation algorithm team. It mainly engaged in the development of AI system for safety and risk. The core technologies include computer vision, Natural Language Processing, biometrics, graphic calculation and anomaly detection and analysis. As of 2018, more than 50 patents have been applied.