In the upper right, we see a colorful rose and the result of three separate filters. Each one of them dramatically modifies the coloring, brightness, and saturation. One way we could learn a filter is through a 3D LUT (Look Up Table), that is to explicitly write down how every RGB pixel transforms. While that would work, using the full 8-bit space for each color channel, it requires 3*(2⁸)³ = 50,331,648 bits to store the whole table for every filter! If we quantize the LUT by only taking some of the of mappings and interpolating between them, the resulting image often has undesired banding issues due to quantization. Instead, let’s learn a transform. In a transform, the goal is to learn a function that takes in a pixel’s color and maps it to another color. It pays off too, the resulting models instafilter uses are only 9K bits, which is only about 300 32-bit numbers! To get started, we need to build a training set and a model.

在右上方，我们看到了五颜六色的玫瑰和三个独立滤镜的结果。 它们中的每一个都极大地改变了色彩，亮度和饱和度。 我们学习过滤器的一种方法是通过3D LUT (查找表)，即明确写下每个RGB像素如何转换。 尽管这可行，但为每个颜色通道使用完整的8位空间，则需要3 *(2⁸)³= 50,331,648位才能为每个滤镜存储整个表！ 如果我们仅通过获取一些映射并在它们之间进行插值来量化LUT，则由于量化，生成的图像通常会出现不希望的条纹问题。 相反，让我们学习一个转换。 在转换中，目标是学习一种接受像素颜色并将其映射到另一种颜色的函数。 它也有回报，instafilter使用的结果模型只有9K位，这仅是300个32位数字！ 首先，我们需要构建训练集和模型。

To build the training set, I took the “Untouched” image as a reference photo. This comes from a project from József Fejes whose goal was to create an image with every possible RGB color represented. In theory, we could use a boring grid of pixels, this colorful rose allows us to visualize the results. Using the high resolution photo, I ran it through each filter on Instagram and saved the result. The result isn’t perfect however, since Instagram downsamples the image and saves it as a JPEG. In practice however, the resulting difference is small enough that the end result looks as good as the app.

为了建立训练集，我将“未触动”图像作为参考照片。 这是来自从项目的József费耶什，其目标是与每一个可能的RGB颜色表示创建一个图像。 从理论上讲，我们可以使用无聊的像素网格，这种彩色玫瑰使我们可以可视化结果。 使用高分辨率照片，我将其运行在Instagram上的每个滤镜中，并保存了结果。 但是结果并不完美，因为Instagram对图像进行了下采样并将其另存为JPEG。 但是，实际上，结果差异很小，以至于最终结果看起来与应用程序一样好。

My first thought for the training set was to train a transform:

对于训练集，我的第一个想法是训练一个转换：

f(R, G, B) -> (R’, G’, B’)

f(R，G，B)->(R'，G'，B')

that is, given the Red, Green, and Blue color channels, learn a function that would map them to the new RGB colors. This didn’t work as well as I wanted, so I had to help out the network. Although the network trained with just RGB, it had to be much larger than I would have wanted to converge. Instead I used

也就是说，给定红色，绿色和蓝色通道，可以学习将它们映射到新的RGB颜色的功能。 这没有我想要的那么好，所以我不得不帮助网络。 尽管网络仅使用RGB进行训练，但它必须比我想要融合的要大得多。 相反，我用

f(R, G, B, L, S) -> (R’, G’, B’)

f(R，G，B，L，S)->(R'，G'，B')

where L and S are the lightness and saturation. It’s simple to calculate these values from the RGB but it helped the network tremendously by doing some of the work upfront. This is the art of machine learning; I could see that Instagram filters modify lightness and saturation so I fed them into the input features. By giving it some additional information the network did not have to waste capacity learning those features itself.

其中L和S是亮度和饱和度。 从RGB计算这些值很简单，但是通过做一些前期工作极大地帮助了网络。 这是机器学习的艺术。 我可以看到Instagram滤镜修改了亮度和饱和度，因此我将它们输入到输入功能中。 通过为其提供一些附加信息，网络不必浪费自身学习这些功能的能力。

The model itself is really simple, a 4 layer fully — connected neural network:

该模型本身非常简单，是一个四层全连接神经网络：

This was my first time really using pytorch and I enjoyed how natural it seemd. It was really freeing to observe the variables and interact with them like regular python objects. If you’ve only ever used tensorflow, give it a try for your next project. To train the model, I borrowed ideas from fast.ai and used one-cycle with a learning rate finder. If you’re interested, you can see the training code directly.

这是我第一次真正使用pytorch，我喜欢它看起来多么自然。 观察变量并像普通的python对象一样与它们进行交互真的很自由。 如果您只使用过tensorflow，请尝试下一个项目。 为了训练模型，我借鉴了fast.ai的想法，并使用一个周期的学习率查找器。 如果您有兴趣，可以直接查看培训代码 。

Training took about 3 minutes per model and I reached a mean average loss of about 0.2. It’s not perfect, mostly due to the imperfections of my training set, but it doesn’t need to be. As far as I can tell, the output is indistinguishable from the real filters.

每个模型的培训时间约为3分钟，而我的平均平均损失约为0.2。 它不是完美的，主要是由于我的训练集的不完善，但这不是必须的。 据我所知，输出与真实过滤器是无法区分的。

It’s easy to use the python module yourself! First install it with pip

自己轻松使用python模块！ 首先用pip安装它

and now you can use it in your own code like this:

现在您可以在自己的代码中使用它，如下所示：

I’ll close with a few more examples:

我将以其他一些示例结束：