[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"similar-guillaume-chevalier--seq2seq-signal-prediction":3,"tool-guillaume-chevalier--seq2seq-signal-prediction":61},[4,18,26,36,44,53],{"id":5,"name":6,"github_repo":7,"description_zh":8,"stars":9,"difficulty_score":10,"last_commit_at":11,"category_tags":12,"status":17},4358,"openclaw","openclaw\u002Fopenclaw","OpenClaw 是一款专为个人打造的本地化 AI 助手，旨在让你在自己的设备上拥有完全可控的智能伙伴。它打破了传统 AI 助手局限于特定网页或应用的束缚，能够直接接入你日常使用的各类通讯渠道，包括微信、WhatsApp、Telegram、Discord、iMessage 等数十种平台。无论你在哪个聊天软件中发送消息，OpenClaw 都能即时响应，甚至支持在 macOS、iOS 和 Android 设备上进行语音交互，并提供实时的画布渲染功能供你操控。\n\n这款工具主要解决了用户对数据隐私、响应速度以及“始终在线”体验的需求。通过将 AI 部署在本地，用户无需依赖云端服务即可享受快速、私密的智能辅助，真正实现了“你的数据，你做主”。其独特的技术亮点在于强大的网关架构，将控制平面与核心助手分离，确保跨平台通信的流畅性与扩展性。\n\nOpenClaw 非常适合希望构建个性化工作流的技术爱好者、开发者，以及注重隐私保护且不愿被单一生态绑定的普通用户。只要具备基础的终端操作能力（支持 macOS、Linux 及 Windows WSL2），即可通过简单的命令行引导完成部署。如果你渴望拥有一个懂你",349277,3,"2026-04-06T06:32:30",[13,14,15,16],"Agent","开发框架","图像","数据工具","ready",{"id":19,"name":20,"github_repo":21,"description_zh":22,"stars":23,"difficulty_score":10,"last_commit_at":24,"category_tags":25,"status":17},3808,"stable-diffusion-webui","AUTOMATIC1111\u002Fstable-diffusion-webui","stable-diffusion-webui 是一个基于 Gradio 构建的网页版操作界面，旨在让用户能够轻松地在本地运行和使用强大的 Stable Diffusion 图像生成模型。它解决了原始模型依赖命令行、操作门槛高且功能分散的痛点，将复杂的 AI 绘图流程整合进一个直观易用的图形化平台。\n\n无论是希望快速上手的普通创作者、需要精细控制画面细节的设计师，还是想要深入探索模型潜力的开发者与研究人员，都能从中获益。其核心亮点在于极高的功能丰富度：不仅支持文生图、图生图、局部重绘（Inpainting）和外绘（Outpainting）等基础模式，还独创了注意力机制调整、提示词矩阵、负向提示词以及“高清修复”等高级功能。此外，它内置了 GFPGAN 和 CodeFormer 等人脸修复工具，支持多种神经网络放大算法，并允许用户通过插件系统无限扩展能力。即使是显存有限的设备，stable-diffusion-webui 也提供了相应的优化选项，让高质量的 AI 艺术创作变得触手可及。",162132,"2026-04-05T11:01:52",[14,15,13],{"id":27,"name":28,"github_repo":29,"description_zh":30,"stars":31,"difficulty_score":32,"last_commit_at":33,"category_tags":34,"status":17},1381,"everything-claude-code","affaan-m\u002Feverything-claude-code","everything-claude-code 是一套专为 AI 编程助手（如 Claude Code、Codex、Cursor 等）打造的高性能优化系统。它不仅仅是一组配置文件，而是一个经过长期实战打磨的完整框架，旨在解决 AI 代理在实际开发中面临的效率低下、记忆丢失、安全隐患及缺乏持续学习能力等核心痛点。\n\n通过引入技能模块化、直觉增强、记忆持久化机制以及内置的安全扫描功能，everything-claude-code 能显著提升 AI 在复杂任务中的表现，帮助开发者构建更稳定、更智能的生产级 AI 代理。其独特的“研究优先”开发理念和针对 Token 消耗的优化策略，使得模型响应更快、成本更低，同时有效防御潜在的攻击向量。\n\n这套工具特别适合软件开发者、AI 研究人员以及希望深度定制 AI 工作流的技术团队使用。无论您是在构建大型代码库，还是需要 AI 协助进行安全审计与自动化测试，everything-claude-code 都能提供强大的底层支持。作为一个曾荣获 Anthropic 黑客大奖的开源项目，它融合了多语言支持与丰富的实战钩子（hooks），让 AI 真正成长为懂上",140436,2,"2026-04-05T23:32:43",[14,13,35],"语言模型",{"id":37,"name":38,"github_repo":39,"description_zh":40,"stars":41,"difficulty_score":32,"last_commit_at":42,"category_tags":43,"status":17},2271,"ComfyUI","Comfy-Org\u002FComfyUI","ComfyUI 是一款功能强大且高度模块化的视觉 AI 引擎，专为设计和执行复杂的 Stable Diffusion 图像生成流程而打造。它摒弃了传统的代码编写模式，采用直观的节点式流程图界面，让用户通过连接不同的功能模块即可构建个性化的生成管线。\n\n这一设计巧妙解决了高级 AI 绘图工作流配置复杂、灵活性不足的痛点。用户无需具备编程背景，也能自由组合模型、调整参数并实时预览效果，轻松实现从基础文生图到多步骤高清修复等各类复杂任务。ComfyUI 拥有极佳的兼容性，不仅支持 Windows、macOS 和 Linux 全平台，还广泛适配 NVIDIA、AMD、Intel 及苹果 Silicon 等多种硬件架构，并率先支持 SDXL、Flux、SD3 等前沿模型。\n\n无论是希望深入探索算法潜力的研究人员和开发者，还是追求极致创作自由度的设计师与资深 AI 绘画爱好者，ComfyUI 都能提供强大的支持。其独特的模块化架构允许社区不断扩展新功能，使其成为当前最灵活、生态最丰富的开源扩散模型工具之一，帮助用户将创意高效转化为现实。",107662,"2026-04-03T11:11:01",[14,15,13],{"id":45,"name":46,"github_repo":47,"description_zh":48,"stars":49,"difficulty_score":10,"last_commit_at":50,"category_tags":51,"status":17},4292,"Deep-Live-Cam","hacksider\u002FDeep-Live-Cam","Deep-Live-Cam 是一款专注于实时换脸与视频生成的开源工具，用户仅需一张静态照片，即可通过“一键操作”实现摄像头画面的即时变脸或制作深度伪造视频。它有效解决了传统换脸技术流程繁琐、对硬件配置要求极高以及难以实时预览的痛点，让高质量的数字内容创作变得触手可及。\n\n这款工具不仅适合开发者和技术研究人员探索算法边界，更因其极简的操作逻辑（仅需三步：选脸、选摄像头、启动），广泛适用于普通用户、内容创作者、设计师及直播主播。无论是为了动画角色定制、服装展示模特替换，还是制作趣味短视频和直播互动，Deep-Live-Cam 都能提供流畅的支持。\n\n其核心技术亮点在于强大的实时处理能力，支持口型遮罩（Mouth Mask）以保留使用者原始的嘴部动作，确保表情自然精准；同时具备“人脸映射”功能，可同时对画面中的多个主体应用不同面孔。此外，项目内置了严格的内容安全过滤机制，自动拦截涉及裸露、暴力等不当素材，并倡导用户在获得授权及明确标注的前提下合规使用，体现了技术发展与伦理责任的平衡。",88924,"2026-04-06T03:28:53",[14,15,13,52],"视频",{"id":54,"name":55,"github_repo":56,"description_zh":57,"stars":58,"difficulty_score":32,"last_commit_at":59,"category_tags":60,"status":17},3704,"NextChat","ChatGPTNextWeb\u002FNextChat","NextChat 是一款轻量且极速的 AI 助手，旨在为用户提供流畅、跨平台的大模型交互体验。它完美解决了用户在多设备间切换时难以保持对话连续性，以及面对众多 AI 模型不知如何统一管理的痛点。无论是日常办公、学习辅助还是创意激发，NextChat 都能让用户随时随地通过网页、iOS、Android、Windows、MacOS 或 Linux 端无缝接入智能服务。\n\n这款工具非常适合普通用户、学生、职场人士以及需要私有化部署的企业团队使用。对于开发者而言，它也提供了便捷的自托管方案，支持一键部署到 Vercel 或 Zeabur 等平台。\n\nNextChat 的核心亮点在于其广泛的模型兼容性，原生支持 Claude、DeepSeek、GPT-4 及 Gemini Pro 等主流大模型，让用户在一个界面即可自由切换不同 AI 能力。此外，它还率先支持 MCP（Model Context Protocol）协议，增强了上下文处理能力。针对企业用户，NextChat 提供专业版解决方案，具备品牌定制、细粒度权限控制、内部知识库整合及安全审计等功能，满足公司对数据隐私和个性化管理的高标准要求。",87618,"2026-04-05T07:20:52",[14,35],{"id":62,"github_repo":63,"name":64,"description_en":65,"description_zh":66,"ai_summary_zh":67,"readme_en":68,"readme_zh":69,"quickstart_zh":70,"use_case_zh":71,"hero_image_url":72,"owner_login":73,"owner_name":74,"owner_avatar_url":75,"owner_bio":76,"owner_company":77,"owner_location":78,"owner_email":79,"owner_twitter":77,"owner_website":77,"owner_url":80,"languages":81,"stars":90,"forks":91,"last_commit_at":92,"license":93,"difficulty_score":32,"env_os":94,"env_gpu":95,"env_ram":94,"env_deps":96,"category_tags":103,"github_topics":104,"view_count":32,"oss_zip_url":77,"oss_zip_packed_at":77,"status":17,"created_at":109,"updated_at":110,"faqs":111,"releases":152},4323,"guillaume-chevalier\u002Fseq2seq-signal-prediction","seq2seq-signal-prediction","Signal forecasting with a Sequence-to-Sequence (seq2seq) Recurrent Neural Network (RNN) model in TensorFlow - Guillaume Chevalier","seq2seq-signal-prediction 是一个基于 TensorFlow 构建的开源教学项目，旨在帮助开发者掌握如何使用序列到序列（seq2seq）循环神经网络（RNN）进行时间序列信号预测。该项目通过四个难度递增的实战练习，引导用户从基础的多维信号预测入手，逐步挑战更复杂的波形叠加预测任务，最终理解编码器 - 解码器架构的核心原理。\n\n它主要解决了初学者在学习深度学习时序预测时缺乏系统性代码实践的问题。不同于仅提供理论讲解的资料，seq2seq-signal-prediction 提供了完整的 Jupyter Notebook 环境和 Google Colab 支持，让用户能够直接运行、修改并观察模型超参数及架构变化对预测结果的影响。其独特的技术亮点在于将抽象的 seq2seq 模型具象化为可视化的信号预测任务，并涵盖了多维数据联合预测（如同时预测多种股票或货币走势）的实际应用场景。\n\n这款工具非常适合具有一定 RNN 基础知识、希望深入理解 seq2seq 架构内部机制的 AI 开发者、数据科学家及高校研究人员。对于想要从理论迈向实战，特别是关注自然语言处理（NLP）之","seq2seq-signal-prediction 是一个基于 TensorFlow 构建的开源教学项目，旨在帮助开发者掌握如何使用序列到序列（seq2seq）循环神经网络（RNN）进行时间序列信号预测。该项目通过四个难度递增的实战练习，引导用户从基础的多维信号预测入手，逐步挑战更复杂的波形叠加预测任务，最终理解编码器 - 解码器架构的核心原理。\n\n它主要解决了初学者在学习深度学习时序预测时缺乏系统性代码实践的问题。不同于仅提供理论讲解的资料，seq2seq-signal-prediction 提供了完整的 Jupyter Notebook 环境和 Google Colab 支持，让用户能够直接运行、修改并观察模型超参数及架构变化对预测结果的影响。其独特的技术亮点在于将抽象的 seq2seq 模型具象化为可视化的信号预测任务，并涵盖了多维数据联合预测（如同时预测多种股票或货币走势）的实际应用场景。\n\n这款工具非常适合具有一定 RNN 基础知识、希望深入理解 seq2seq 架构内部机制的 AI 开发者、数据科学家及高校研究人员。对于想要从理论迈向实战，特别是关注自然语言处理（NLP）之外时序预测应用的工程师来说，这是一个极佳的入门与进阶资源。","\n# [Sequence to Sequence (seq2seq) Recurrent Neural Network (RNN) for Time Series Forecasting](https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002Fseq2seq-signal-prediction)\n\n***Note: You can find here the accompanying [seq2seq RNN forecasting presentation's slides](https:\u002F\u002Fdrive.google.com\u002Fdrive\u002Ffolders\u002F1U0xQMxVespjQilMhYW4mDxN02IwEW67I), as well as the Google Colab file for running the present notebook (if you're not already in Colab).***\n\nThis is a series of exercises that you can try to solve to learn how to code Encoder-Decoder Sequence to Sequence Recurrent Neural Networks (seq2seq RNNs). You can solve different simple toy signal prediction problems. Seq2seq architectures may also be used for other sophisticated purposes, such as for Natural Language Processing (NLP). \n\nIn this project are given 4 exercises of gradually increasing difficulty. I take for granted that you have at least some knowledge of how RNN works and how can they be shaped into an encoder and a decoder seq2seq setup of the most simple form (without attention). To learn more about RNNs in TensorFlow, you may want to visit [this other RNN project](https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002FLSTM-Human-Activity-Recognition) which I have built for that.\n\nThe current project is a series of example I have first built in French, but I haven't got the time to generate all the charts anew with proper English text. I have built this project at first for the practical part of the third hour of a [master class](https:\u002F\u002Fwebaquebec.org\u002Fclasses-de-maitre\u002Fdeep-learning-avec-tensorflow) conference I presented at the Web At Quebec (WAQ), originally in March 2017.\n\n## How to use this \".ipynb\" Python notebook ?\n\nI made available an \".py\" Python version of this tutorial within the [repository](https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002Fseq2seq-signal-prediction), but it's more convenient to run the code inside the notebook or within Google Colab.\n\nFor running the notebook, you can run `jupyter-notebook` in the command-line to launch the web notebook IDE, and choose the `.ipynb` file. For Google Colab, if you want to run the code using GPU, make sure to do `Runtime > Change Runtime Type` and to select `GPU` for `Python 3`.\n\n## Exercises\n\nNote that the dataset changes in function of the exercice. Most of the time, you will have to edit the neural networks' training parameter to succeed in doing the exercise, but at a certain point, changes in the architecture itself will be asked and required. The datasets used for this exercises are found in [`datasets.py`](https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002Fseq2seq-signal-prediction\u002Fblob\u002Fmaster\u002Fdatasets.py).\n\n### Exercise 1\n\nIn theory, it is possible to create a perfect prediction of the signal for this exercise as it is deterministic. The neural network's parameters has been set to \"somehow\" acceptable values for a first training. You'll want to play with the hyperparameters until you reach predictions like those:\n\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_8b46a99e555c.png\" \u002F>\n\nNote: the neural network sees only what is to the left of the chart and is trained to predict what is at the right (predictions in yellow). \n\nWe have 2 time series at once to predict, which are tied together. That means our neural network processes multidimensional data. A simple example would be to receive as an argument the past values of multiple stock market symbols in order to predict the future values of all those symbols with the neural network, which values are evolving together in time. That is what we will do in the exercise 4 with USD and EUR values of the BTC that we'll see both at once. \n\n\n### Exercise 2\n\nHere, rather than 2 signals in parallel to predict, we have only one, for simplicity. HOWEVER, this signal is a superposition of two sine waves of varying wavelenght and offset (and restricted to a particular min and max limit of wavelengts). \n\nIn order to finish this exercise properly, you will need to edit the neural network's hyperparameters. We would recommend first trying with hyperparameters like those:\n\n- `n_samples = 125000`\n- `epochs = 1`\n- `batch_size = 50`\n- `hidden_dim = 35`\n\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_b21932ed5500.png\" \u002F>\n\nHere are predictions achieved with a bigger neural networks with 3 stacked recurrent cells and a width of 500 hidden units for each of those cells: \n\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_f7ca6ac96ad2.png\" \u002F>\n\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_7db3277e746a.png\" \u002F>\n\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_7db3277e746a.png\" \u002F>\n\nNote that it would be possible to obtain better results with a smaller neural network, provided better training hyperparameters and a longer training, adding dropout and a few things, and on. \n\n### Exercise 3\n\nThis exercise is similar to the previous one, except that the input data given to the encoder is noisy. The expected output is NOT noisy. This makes the task a bit harder. In this specific data context, we can call our neuralnetwork a denoising autoregressive autoencoder. Here is a good example of what a training example (and a prediction) could now looks like :\n\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_854639a1653f.png\" \u002F>\n\nTherefore the neural network is brought to denoise the signal to interpret its future smooth values. Here are some example of better predictions on this version of the dataset :\n\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_60d1f51cb9b8.png\" \u002F>\n\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_3400da8e93f5.png\" \u002F>\n\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_02d9ae19b8e1.png\" \u002F>\n\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_d1075e4bc0c0.png\" \u002F>\n\nSimilarly as I said for the exercise 2, it would be possible here too to obtain better results. Note that it would also have been possible to ask you to predict to reconstruct the denoised signal from the noisy input (rather than trying to predict the future values of it) as a denoising autoencoder. This type of architecture is also useful for data compression, such as manipulating images, for instance.\n\n### Exercise 4\n\nThis exercise is much harder than the previous ones and is built more as an open-ended suggestion. It is to predict the future value of the Bitcoin's price. We have here some daily market data of the bitcoin's value, that is, BTC\u002FUSD and BTC\u002FEUR. This is not enough to build a good predictor - at least having data precise at the minute level, or second level, would be more interesting. Here is a prediction that was made on the actual future values, the neural network has not been trained on the future values shown here so this is a legitimate prediction, given a well-enough model trained on the task: \n\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_afe85f096617.png\" \u002F>\n\nDisclaimer: this prediction of the future values was really good and you should not expect predictions to be always that good using as few data as actually (side note: the other prediction charts in this project are all \"average\" except this one). I mostly didn't really took the time to compare this model to other financial models. For this exercise, you can try to plug more valuable financial data into the model in order to make more accurate predictions. Let me remind you that I provided the code for the datasets in [`datasets.py`](https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002Fseq2seq-signal-prediction\u002Fblob\u002Fmaster\u002Fdatasets.py), but that could be replaced with more comprehensive data for predicting more accurately the Bitcoin. \n\nThe input and output dimensions of the model is 2D accepts (BTC\u002FUSD and BTC\u002FEUR). As an example, you could create additionnal input dimensions\u002Fstreams which could contain meteo data and more financial data, such as the S&P 500, the Dow Jones, and so on. Other more creative input data could be sine waves (or other-type-shaped waves such as saw waves or triangles or two signals for `cos` and `sin`) representing the fluctuation of minutes, hours, days, weeks, months, years, moon cycles, and on (as we did in [Neuraxio's Time Series Solution](https:\u002F\u002Fwww.neuraxio.com\u002Fen\u002Ftime-series-solution)). This could be combined with a stream of social media [sentiment analysis](https:\u002F\u002Fgithub.com\u002FNeuraxio\u002FSentiment-Analysis-AutoML) about the word \"Bitcoin\" to have another input signal which is more human-based and abstract. It is also interesting to know [where is the bitcoin most used](http:\u002F\u002Fimages.google.com\u002Fsearch?tbm=isch&q=bitcoin+heatmap+world).\n\nWith all the above-mentionned examples, it would be possible to have all of this as input features, at every time steps: (BTC\u002FUSD, BTC\u002FEUR, Dow_Jones, SP_500, hour_of_day, day_of_week, day_of_month, week_of_year, year, moon_cycle, meteo_USA, meteo_EUROPE, social_sentiment). Finally, there could be those two output features, or more: (BTC\u002FUSD, BTC\u002FEUR).\n\nThis prediction concept and similar time series forecasting algorithms can apply to many many things, such as auto-correcting machines for Industry 4.0, quality assurance in production chains, traffic forecast, meteo prediction, movements and action prediction, and lots of other types of shot-term and mid-term statistical predictions or forecasts.\n\n## Install Requirements\n\n\n```python\n!pip install tensorflow-gpu==2.1 neuraxle==0.3.1 neuraxle_tensorflow==0.1.0\n```\n\n    Requirement already satisfied: tensorflow-gpu==2.1 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (2.1.0)\n    Requirement already satisfied: neuraxle==0.3.1 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (0.3.1)\n    Requirement already satisfied: neuraxle_tensorflow==0.1.0 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (0.1.0)\n    Requirement already satisfied: google-pasta>=0.1.6 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from tensorflow-gpu==2.1) (0.1.8)\n    Requirement already satisfied: opt-einsum>=2.3.2 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from tensorflow-gpu==2.1) (3.1.0)\n    Requirement already satisfied: tensorboard\u003C2.2.0,>=2.1.0 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from tensorflow-gpu==2.1) (2.1.0)\n    Requirement already satisfied: gast==0.2.2 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from tensorflow-gpu==2.1) (0.2.2)\n    Requirement already satisfied: protobuf>=3.8.0 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from tensorflow-gpu==2.1) (3.10.0)\n    Requirement already satisfied: termcolor>=1.1.0 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from tensorflow-gpu==2.1) (1.1.0)\n    Requirement already satisfied: wrapt>=1.11.1 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from tensorflow-gpu==2.1) (1.11.2)\n    Requirement already satisfied: absl-py>=0.7.0 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from tensorflow-gpu==2.1) (0.9.0)\n    Requirement already satisfied: six>=1.12.0 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from tensorflow-gpu==2.1) (1.12.0)\n    Requirement already satisfied: keras-applications>=1.0.8 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from tensorflow-gpu==2.1) (1.0.8)\n    Requirement already satisfied: tensorflow-estimator\u003C2.2.0,>=2.1.0rc0 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from tensorflow-gpu==2.1) (2.1.0)\n    Requirement already satisfied: astor>=0.6.0 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from tensorflow-gpu==2.1) (0.8.1)\n    Requirement already satisfied: keras-preprocessing>=1.1.0 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from tensorflow-gpu==2.1) (1.1.0)\n    Requirement already satisfied: numpy\u003C2.0,>=1.16.0 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from tensorflow-gpu==2.1) (1.17.5)\n    Requirement already satisfied: wheel>=0.26; python_version >= \"3\" in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from tensorflow-gpu==2.1) (0.33.6)\n    Requirement already satisfied: grpcio>=1.8.6 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from tensorflow-gpu==2.1) (1.15.0)\n    Requirement already satisfied: scipy==1.4.1; python_version >= \"3\" in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from tensorflow-gpu==2.1) (1.4.1)\n    Requirement already satisfied: matplotlib in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from neuraxle==0.3.1) (3.1.2)\n    Requirement already satisfied: Flask-RESTful>=0.3.7 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from neuraxle==0.3.1) (0.3.7)\n    Requirement already satisfied: conv==0.2 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from neuraxle==0.3.1) (0.2)\n    Requirement already satisfied: Flask>=1.1.1 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from neuraxle==0.3.1) (1.1.1)\n    Requirement already satisfied: joblib>=0.13.2 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from neuraxle==0.3.1) (0.14.1)\n    Requirement already satisfied: scikit-learn>=0.20.3 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from neuraxle==0.3.1) (0.22.1)\n    Requirement already satisfied: werkzeug>=0.11.15 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1) (0.16.0)\n    Requirement already satisfied: google-auth\u003C2,>=1.6.3 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1) (1.10.1)\n    Requirement already satisfied: google-auth-oauthlib\u003C0.5,>=0.4.1 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1) (0.4.1)\n    Requirement already satisfied: markdown>=2.6.8 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1) (3.1.1)\n    Requirement already satisfied: setuptools>=41.0.0 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1) (42.0.2)\n    Requirement already satisfied: requests\u003C3,>=2.21.0 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1) (2.21.0)\n    Requirement already satisfied: h5py in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from keras-applications>=1.0.8->tensorflow-gpu==2.1) (2.8.0)\n    Requirement already satisfied: kiwisolver>=1.0.1 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from matplotlib->neuraxle==0.3.1) (1.1.0)\n    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from matplotlib->neuraxle==0.3.1) (2.4.6)\n    Requirement already satisfied: python-dateutil>=2.1 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from matplotlib->neuraxle==0.3.1) (2.6.1)\n    Requirement already satisfied: cycler>=0.10 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from matplotlib->neuraxle==0.3.1) (0.10.0)\n    Requirement already satisfied: aniso8601>=0.82 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from Flask-RESTful>=0.3.7->neuraxle==0.3.1) (8.0.0)\n    Requirement already satisfied: pytz in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from Flask-RESTful>=0.3.7->neuraxle==0.3.1) (2018.9)\n    Requirement already satisfied: click>=5.1 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from Flask>=1.1.1->neuraxle==0.3.1) (7.0)\n    Requirement already satisfied: Jinja2>=2.10.1 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from Flask>=1.1.1->neuraxle==0.3.1) (2.10.3)\n    Requirement already satisfied: itsdangerous>=0.24 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from Flask>=1.1.1->neuraxle==0.3.1) (1.1.0)\n    Requirement already satisfied: rsa\u003C4.1,>=3.1.4 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from google-auth\u003C2,>=1.6.3->tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1) (4.0)\n    Requirement already satisfied: pyasn1-modules>=0.2.1 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from google-auth\u003C2,>=1.6.3->tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1) (0.2.7)\n    Requirement already satisfied: cachetools\u003C5.0,>=2.0.0 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from google-auth\u003C2,>=1.6.3->tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1) (4.0.0)\n    Requirement already satisfied: requests-oauthlib>=0.7.0 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from google-auth-oauthlib\u003C0.5,>=0.4.1->tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1) (1.3.0)\n    Requirement already satisfied: chardet\u003C3.1.0,>=3.0.2 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from requests\u003C3,>=2.21.0->tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1) (3.0.4)\n    Requirement already satisfied: urllib3\u003C1.25,>=1.21.1 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from requests\u003C3,>=2.21.0->tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1) (1.24.3)\n    Requirement already satisfied: certifi>=2017.4.17 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from requests\u003C3,>=2.21.0->tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1) (2019.11.28)\n    Requirement already satisfied: idna\u003C2.9,>=2.5 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from requests\u003C3,>=2.21.0->tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1) (2.8)\n    Requirement already satisfied: MarkupSafe>=0.23 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from Jinja2>=2.10.1->Flask>=1.1.1->neuraxle==0.3.1) (1.1.1)\n    Requirement already satisfied: pyasn1>=0.1.3 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from rsa\u003C4.1,>=3.1.4->google-auth\u003C2,>=1.6.3->tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1) (0.4.8)\n    Requirement already satisfied: oauthlib>=3.0.0 in \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib\u003C0.5,>=0.4.1->tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1) (3.1.0)\n\n\n\n```python\nimport urllib\n\ndef download_import(filename):\n    with open(filename, \"wb\") as f:\n        # Downloading like that is needed because of Colab operating from a Google Drive folder that is just \"shared with you\".\n        # https:\u002F\u002Fdrive.google.com\u002Fdrive\u002Ffolders\u002F1U0xQMxVespjQilMhYW4mDxN02IwEW67I\n        url = 'https:\u002F\u002Fraw.githubusercontent.com\u002Fguillaume-chevalier\u002Fseq2seq-signal-prediction\u002Fmaster\u002F{}'.format(filename)\n        f.write(urllib.request.urlopen(url).read())\n\ndownload_import(\"datasets.py\")\ndownload_import(\"plotting.py\")\ndownload_import(\"steps.py\")\n```\n\n\n```python\nfrom typing import List\nfrom logging import warning\n\nimport tensorflow as tf\nfrom neuraxle.data_container import DataContainer\nfrom neuraxle.hyperparams.space import HyperparameterSamples\nfrom neuraxle.metaopt.random import ValidationSplitWrapper\nfrom neuraxle.metrics import MetricsWrapper\nfrom neuraxle.pipeline import Pipeline, MiniBatchSequentialPipeline\nfrom neuraxle.steps.data import EpochRepeater, DataShuffler\nfrom neuraxle.steps.flow import TrainOnlyWrapper\nfrom neuraxle.steps.loop import ForEachDataInput\nfrom sklearn.metrics import mean_squared_error\nfrom tensorflow_core.python.client import device_lib\nfrom tensorflow_core.python.keras import Input, Model\nfrom tensorflow_core.python.keras.layers import GRUCell, RNN, Dense\nfrom tensorflow_core.python.training.adam import AdamOptimizer\n\nfrom datasets import generate_data\nfrom datasets import metric_3d_to_2d_wrapper\nfrom neuraxle_tensorflow.tensorflow_v1 import TensorflowV1ModelStep\nfrom neuraxle_tensorflow.tensorflow_v2 import Tensorflow2ModelStep\nfrom plotting import plot_metrics\nfrom steps import MeanStdNormalizer, ToNumpy, PlotPredictionsWrapper\n\n%matplotlib inline\n```\n\n\n```python\ndef choose_tf_device():\n    \"\"\"\n    Choose a TensorFlow device (e.g.: GPU if available) to compute on.\n    \"\"\"\n    tf.debugging.set_log_device_placement(True)\n    devices = [x.name for x in device_lib.list_local_devices()]\n    print('You can use the following tf devices: {}'.format(devices))\n    try:\n        chosen_device = [d for d in devices if 'gpu' in d.lower()][0]\n    except:\n        warning(\n            \"No GPU device found. Please make sure to do `Runtime > Change Runtime Type` and select GPU for Python 3.\")\n        chosen_device = devices[0]\n    print('Chosen Device: {}'.format(chosen_device))\n    return chosen_device\n\nchosen_device = choose_tf_device()\n```\n\n    You can use the following tf devices: ['\u002Fdevice:CPU:0', '\u002Fdevice:XLA_CPU:0', '\u002Fdevice:XLA_GPU:0', '\u002Fdevice:GPU:0']\n    Chosen Device: \u002Fdevice:XLA_GPU:0\n\n\n\n## Definition of the Neural Architecture\n\n### Basic Sequence To Sequence (seq2seq) RNN\n\nHere is a basic sequence to sequence neural architecture. \"ABC\" is a past input. \"WXYZ\" is here both a future output and a future input as a feedback loop. This feedback loop has been proven to improve the results of RNNs in some cases ([read more](https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002FAwesome-Deep-Learning-Resources#recurrent-neural-networks)). \n\n\u003Cimg src=\"https:\u002F\u002Fwww.tensorflow.org\u002Fimages\u002Fbasic_seq2seq.png\" \u002F>\n\nIn our case, we won't do such a feedback loop, as it requires more complex sampling during training and testing and would be too complicated for today's practical example.\n\n### Our Stacked GRU seq2seq RNN\n\nHere is what we do. The \"H\" is the hidden output of the encoder RNN's last time step. We replicate this value across time in the future as a future data input to the RNN to make it remember the context of the present at all times when predicting the future.\n\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_fde9af34637f.png\" \u002F>\n\n Notice that we could have instead plugged an attention mechanism here. Doing so would allow the neural net to re-analyze the past at every step in the future if it needed. Attention mechanisms would be more useful in contexts of Machine Translation (MT), where it's sometimes important to go see back \"word per word\" what was written, rather than being limited by our short term memory that was accumulated once after reading everything, for instance. More recent Machine Translation approaches like BERT ([read on BERT](https:\u002F\u002Fwww.umaneo.com\u002Fpost\u002Fa-review-of-recent-natural-language-processing-approaches) \u002F [see example of using BERT](https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002FReuBERT)) only uses attention mechanisms without RNNs (with some tradeoffs, however).\n\n\n## Creating Tensorflow 2 Model\n\nLet's proceed and code what we see in the image just above.\n\n\n\n```python\ndef create_model(step: Tensorflow2ModelStep) -> tf.keras.Model:\n    \"\"\"\n   Create a TensorFlow v2 sequence to sequence (seq2seq) encoder-decoder model.\n\n   :param step: The base Neuraxle step for TensorFlow v2 (Tensorflow2ModelStep)\n   :return: TensorFlow v2 Keras model\n    \"\"\"\n    # shape: (batch_size, seq_length, input_dim)\n    encoder_inputs = Input(\n        shape=(None, step.hyperparams['input_dim']),\n        batch_size=None,\n        dtype=tf.dtypes.float32,\n        name='encoder_inputs'\n    )\n\n    last_encoder_outputs, last_encoders_states = _create_encoder(step, encoder_inputs)\n    decoder_outputs = _create_decoder(step, last_encoder_outputs, last_encoders_states)\n\n    return Model(encoder_inputs, decoder_outputs)\n\ndef _create_encoder(step: Tensorflow2ModelStep, encoder_inputs: Input) -> (tf.Tensor, List[tf.Tensor]):\n    \"\"\"\n   Create an encoder RNN using GRU Cells. GRU cells are similar to LSTM cells.\n\n   :param step: The base Neuraxle step for TensorFlow v2 (class Tensorflow2ModelStep)\n    :param encoder_inputs: encoder inputs layer of shape (batch_size, seq_length, input_dim)\n    :return: (last encoder outputs, last stacked encoders states)\n                last_encoder_outputs shape: (batch_size, hidden_dim)\n                last_encoder_states shape: (layers_stacked_count, batch_size, hidden_dim)\n    \"\"\"\n    encoder = RNN(cell=_create_stacked_rnn_cells(step), return_sequences=False, return_state=True)\n\n    last_encoder_outputs_and_states = encoder(encoder_inputs)\n    # last_encoder_outputs shape: (batch_size, hidden_dim)\n    # last_encoder_states shape: (layers_stacked_count, batch_size, hidden_dim)\n\n    # refer to: https:\u002F\u002Fwww.tensorflow.org\u002Fapi_docs\u002Fpython\u002Ftf\u002Fkeras\u002Flayers\u002FRNN?version=stable#output_shape_2\n    last_encoder_outputs, *last_encoders_states = last_encoder_outputs_and_states\n    return last_encoder_outputs, last_encoders_states\n\ndef _create_decoder(step: Tensorflow2ModelStep, last_encoder_outputs: tf.Tensor, last_encoders_states: List[tf.Tensor]) -> tf.Tensor:\n    \"\"\"\n   Create a decoder RNN using GRU cells.\n\n   :param step: The base Neuraxle step for TensorFlow v2 (Tensorflow2ModelStep)\n    :param last_encoders_states: last encoder states tensor\n    :param last_encoder_outputs: last encoder output tensor\n    :return: decoder output\n    \"\"\"\n    decoder_lstm = RNN(cell=_create_stacked_rnn_cells(step), return_sequences=True, return_state=False)\n\n    last_encoder_output = tf.expand_dims(last_encoder_outputs, axis=1)\n    # last encoder output shape: (batch_size, 1, hidden_dim)\n\n    replicated_last_encoder_output = tf.repeat(\n        input=last_encoder_output,\n        repeats=step.hyperparams['window_size_future'],\n        axis=1\n    )\n    # replicated last encoder output shape: (batch_size, window_size_future, hidden_dim)\n\n    decoder_outputs = decoder_lstm(replicated_last_encoder_output, initial_state=last_encoders_states)\n    # decoder outputs shape: (batch_size, window_size_future, hidden_dim)\n\n    decoder_dense = Dense(step.hyperparams['output_dim'])\n    # decoder outputs shape: (batch_size, window_size_future, output_dim)\n\n    return decoder_dense(decoder_outputs)\n\ndef _create_stacked_rnn_cells(step: Tensorflow2ModelStep) -> List[GRUCell]:\n    \"\"\"\n   Create a `layers_stacked_count` amount of GRU cells and stack them on top of each other.\n   They have a `hidden_dim` number of neuron layer size.\n\n   :param step: The base Neuraxle step for TensorFlow v2 (Tensorflow2ModelStep)\n    :return: list of gru cells\n    \"\"\"\n    cells = []\n    for _ in range(step.hyperparams['layers_stacked_count']):\n        cells.append(GRUCell(step.hyperparams['hidden_dim']))\n\n    return cells\n```\n\n## Create Loss\n\nUsing the Mean Squared Error (MSE) and weight decay (L2 penality) regularization.\n\n\n```python\ndef create_loss(step: Tensorflow2ModelStep, expected_outputs: tf.Tensor, predicted_outputs: tf.Tensor) -> tf.Tensor:\n    \"\"\"\n    Create model loss.\n\n   :param step: The base Neuraxle step for TensorFlow v2 (Tensorflow2ModelStep)\n   :param expected_outputs: expected outputs of shape (batch_size, window_size_future, output_dim)\n   :param predicted_outputs: expected outputs of shape (batch_size, window_size_future, output_dim)\n   :return: loss (a tf Tensor that is a float)\n    \"\"\"\n    l2 = step.hyperparams['lambda_loss_amount'] * sum(\n        tf.reduce_mean(tf.nn.l2_loss(tf_var))\n        for tf_var in step.model.trainable_variables\n    )\n\n    output_loss = sum(\n        tf.reduce_mean(tf.nn.l2_loss(pred - expected))\n        for pred, expected in zip(predicted_outputs, expected_outputs)\n    ) \u002F float(len(predicted_outputs))\n\n    return output_loss + l2\n```\n\n## Create Optimizer\n\nAdam often wins.\n\n\n```python\ndef create_optimizer(step: TensorflowV1ModelStep) -> AdamOptimizer:\n    \"\"\"\n   Create a TensorFlow 2 Optimizer: here the AdamOptimizer.\n\n   :param step: The base Neuraxle step for TensorFlow v2 (Tensorflow2ModelStep)\n    :return: optimizer\n    \"\"\"\n    return AdamOptimizer(learning_rate=step.hyperparams['learning_rate'])\n```\n\n## Generate or Load the Data\n\nTo change which exercise you are doing, change the value of the `exercise_number` variable (that is, the first line in the code cell below):\n\n\n```python\nexercice_number = 1\nprint('exercice {}\\n=================='.format(exercice_number))\n\ndata_inputs, expected_outputs = generate_data(\n    # See: https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002Fseq2seq-signal-prediction\u002Fblob\u002Fmaster\u002Fdatasets.py\n    exercice_number=exercice_number,\n    n_samples=None,\n    window_size_past=None,\n    window_size_future=None\n)\n\nprint('data_inputs shape: {} => (n_samples, window_size_past, input_dim)'.format(data_inputs.shape))\nprint('expected_outputs shape: {} => (n_samples, window_size_future, output_dim)'.format(expected_outputs.shape))\n\nsequence_length = data_inputs.shape[1]\ninput_dim = data_inputs.shape[2]\noutput_dim = expected_outputs.shape[2]\n\nbatch_size = 100\nepochs = 15\nvalidation_size = 0.15\nmax_plotted_validation_predictions = 10\n```\n\n    exercice 1\n    ==================\n    data_inputs shape: (1000, 10, 2) => (n_samples, window_size_past, input_dim)\n    expected_outputs shape: (1000, 10, 2) => (n_samples, window_size_future, output_dim)\n\n\n## Neural Network's hyperparameters\n\n\n```python\nseq2seq_pipeline_hyperparams = HyperparameterSamples({\n    'hidden_dim': 12,\n    'layers_stacked_count': 2,\n    'lambda_loss_amount': 0.0003,\n    'learning_rate': 0.001,\n    'window_size_future': sequence_length,\n    'output_dim': output_dim,\n    'input_dim': input_dim\n})\n\nprint('hyperparams: {}'.format(seq2seq_pipeline_hyperparams))\n```\n\n    hyperparams: HyperparameterSamples([('hidden_dim', 12), ('layers_stacked_count', 2), ('lambda_loss_amount', 0.0003), ('learning_rate', 0.001), ('window_size_future', 10), ('output_dim', 2), ('input_dim', 2)])\n\n\n## The Pipeline\n\nSeeing [dirty Machine Learning code](https:\u002F\u002Fwww.neuraxio.com\u002Fen\u002Fblog\u002Fclean-code\u002F2019\u002F12\u002F26\u002Fmachine-learning-competition-code.html) has almost become the industry norm. And it is for sure contributing to the reasons [why 87% of data science projects never make it into production](https:\u002F\u002Fventurebeat.com\u002F2019\u002F07\u002F19\u002Fwhy-do-87-of-data-science-projects-never-make-it-into-production\u002F).\n\nHere, we use advanced design patterns (pipe and filter) to do what we call [clean machine learning](https:\u002F\u002Fwww.neuraxio.com\u002Fen\u002Fblog\u002Fneuraxle\u002F2019\u002F10\u002F26\u002Fneat-machine-learning-pipelines.html). Those design patterns are inspired of [scikit-learn's pipeline class](https:\u002F\u002Fwww.neuraxio.com\u002Fen\u002Fblog\u002Fscikit-learn\u002F2020\u002F01\u002F03\u002Fwhat-is-wrong-with-scikit-learn.html).\n\n### Defining the Deep Learning Pipeline\n\nHere, we first define the pipeline using a [Tensorflow2ModelStep](https:\u002F\u002Fgithub.com\u002FNeuraxio\u002FNeuraxle-TensorFlow). The MeanStdNormalizer helps us normalize data, as a neural network needs to see normalized data. \n\n\n\n```python\nfeature_0_metric = metric_3d_to_2d_wrapper(mean_squared_error)\nmetrics = {'mse': feature_0_metric}\n\nsignal_prediction_pipeline = Pipeline([\n    ForEachDataInput(MeanStdNormalizer()),\n    ToNumpy(),\n    PlotPredictionsWrapper(Tensorflow2ModelStep(\n        # See: https:\u002F\u002Fgithub.com\u002FNeuraxio\u002FNeuraxle-TensorFlow\n        create_model=create_model,\n        create_loss=create_loss,\n        create_optimizer=create_optimizer,\n        expected_outputs_dtype=tf.dtypes.float32,\n        data_inputs_dtype=tf.dtypes.float32,\n        print_loss=False,\n        device_name=chosen_device\n).set_hyperparams(seq2seq_pipeline_hyperparams))]).set_name('SignalPrediction')\n\n```\n\n### Defining how to Train our Deep Learning Pipeline\n\nFinally, let's wrap the pipeline with an EpochRepeater, ValidationSplitWrapper, DataShuffler, MiniBatchSequentialPipeline and MetricsWrapper to handle all it needs to be trained. You can refer to [Neuraxle's Documentation](https:\u002F\u002Fwww.neuraxle.org\u002Fstable\u002Findex.html) for more info on those objects.\n\n\n```python\n\npipeline = Pipeline([EpochRepeater(\n    ValidationSplitWrapper(\n        MetricsWrapper(Pipeline([\n            TrainOnlyWrapper(DataShuffler()),\n            MiniBatchSequentialPipeline([\n                MetricsWrapper(\n                    signal_prediction_pipeline,\n                    metrics=metrics,\n                    name='batch_metrics'\n                )], batch_size=batch_size)\n            ]), \n            metrics=metrics,\n            name='epoch_metrics',\n            print_metrics=True\n        ),\n        test_size=validation_size,\n        scoring_function=feature_0_metric), \n    epochs=epochs)\n])\n\n```\n\n    \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages\u002Fneuraxle\u002Fpipeline.py:353: UserWarning: Replacing MiniBatchSequentialPipeline[Joiner].batch_size by MiniBatchSequentialPipeline.batch_size.\n      'Replacing {}[{}].batch_size by {}.batch_size.'.format(self.name, step.name, self.name))\n\n\n## Training of the neural net\n\nTime to fit the model on the data.\n\n\n\n```python\n\npipeline, outputs = pipeline.fit_transform(data_inputs, expected_outputs)\n\n```\n\n    Executing op RandomUniform in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op Sub in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op Mul in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op Add in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op VarHandleOp in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op VarIsInitializedOp in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op LogicalNot in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op Assert in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op AssignVariableOp in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op RandomStandardNormal in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op Qr in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op DiagPart in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op Sign in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op Transpose in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op Reshape in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op VarHandleOp in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op Fill in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op Cast in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference_keras_scratch_graph_365 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference_keras_scratch_graph_370 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference_keras_scratch_graph_375 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference_keras_scratch_graph_380 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference_keras_scratch_graph_385 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference_keras_scratch_graph_390 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference_keras_scratch_graph_395 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference_keras_scratch_graph_400 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference_keras_scratch_graph_405 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference_keras_scratch_graph_410 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference_keras_scratch_graph_415 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op VarHandleOp in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op VarHandleOp in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op VarHandleOp in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op AssignVariableOp in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op Shape in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op StridedSlice in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op Unpack in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op StridedSlice in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op ReadVariableOp in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op Unpack in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op MatMul in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op BiasAdd in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op Split in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op SplitV in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op AddV2 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op Sigmoid in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op Tanh in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op Less in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op AddV2 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op Pack in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __forward__defun_call_1537 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __forward__defun_call_1547 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference__defun_call_1557 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference__defun_call_1566 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __forward__defun_call_1580 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __forward__defun_call_1606 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference__defun_call_1617 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __forward__defun_call_1631 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op L2Loss in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op Mean in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op RealDiv in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op BroadcastGradientArgs in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op Sum in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op Neg in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op Tile in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op StridedSliceGrad in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op AddN in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op BiasAddGrad in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op MatMul in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op MatMul in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op InvertPermutation in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op TanhGrad in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op AddN in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op SigmoidGrad in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op ConcatV2 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op Pack in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op AddN in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference___backward__defun_call_1625_1632 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference___backward__defun_call_1589_1607 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference___backward__defun_call_1574_1581 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference___backward__defun_call_1531_1538 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op AddN in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op VarHandleOp in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op ResourceApplyAdam in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference__defun_call_1530 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference__defun_call_1544 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference__defun_call_1573 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference__defun_call_1588 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference__defun_call_1624 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __forward__defun_call_65540 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __forward__defun_call_65550 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __forward__defun_call_65567 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __forward__defun_call_65593 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __forward__defun_call_65610 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op AddN in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference___backward__defun_call_65604_65611 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference___backward__defun_call_65576_65594 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference___backward__defun_call_65561_65568 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference___backward__defun_call_65534_65541 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference__defun_call_65533 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference__defun_call_65547 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference__defun_call_65560 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference__defun_call_65575 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference__defun_call_65603 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    {'mse': 0.18414642847122925}\n    {'mse': 0.1778781379709343}\n    {'mse': 0.1723181842129053}\n    {'mse': 0.1658200688421554}\n    {'mse': 0.1591329577983185}\n    {'mse': 0.15131258011101834}\n    {'mse': 0.1436201535512516}\n    {'mse': 0.1343595503512161}\n    {'mse': 0.12474072112690562}\n    {'mse': 0.11462532630747631}\n    {'mse': 0.10271182130173581}\n    {'mse': 0.0906442166022616}\n    {'mse': 0.07585859336447773}\n    {'mse': 0.06317439259405164}\n    {'mse': 0.04988300184267241}\n    {'mse': 0.041345448752856694}\n    {'mse': 0.034553488508200454}\n    {'mse': 0.03218617403485365}\n    {'mse': 0.02922688138678744}\n    {'mse': 0.02631547230588055}\n    {'mse': 0.022075968214915552}\n    {'mse': 0.018800000904722468}\n    {'mse': 0.01640079469351695}\n    {'mse': 0.014737265865397323}\n    {'mse': 0.013079363146911618}\n    {'mse': 0.01166897820815228}\n    {'mse': 0.010537850442431971}\n    {'mse': 0.00938083864872879}\n    {'mse': 0.008495135058422493}\n    {'mse': 0.007566329717239811}\n\n\n## Visualizing Test Predictions\n\nSee how your training performed.\n\n\n```python\nplot_metrics(pipeline=pipeline, exercice_number=exercice_number)\n```\n\n    last mse train: 0.008495135058422493\n    best mse train: 0.008495135058422493\n    last mse validation: 0.007566329717239811\n    best mse validation: 0.007566329717239811\n\n\n\n![png](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_2df3686cab3f.png)\n\n\n\n```python\ndef plot_predictions(data_inputs, expected_outputs, pipeline, max_plotted_predictions):\n    _, _, data_inputs_validation, expected_outputs_validation = \\\n        pipeline.get_step_by_name('ValidationSplitWrapper').split(data_inputs, expected_outputs)\n\n    pipeline.apply('toggle_plotting')\n    pipeline.apply('set_max_plotted_predictions', max_plotted_predictions)\n\n    signal_prediction_pipeline = pipeline.get_step_by_name('SignalPrediction')\n    signal_prediction_pipeline.transform_data_container(DataContainer(\n        data_inputs=data_inputs_validation,\n        expected_outputs=expected_outputs_validation\n    ))\n\nplot_predictions(data_inputs, expected_outputs, pipeline, max_plotted_validation_predictions)\n```\n\n    Executing op __inference__defun_call_1562783 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference__defun_call_1562789 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference__defun_call_1562798 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference__defun_call_1562805 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n    Executing op __inference__defun_call_1562813 in device \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0\n\n\n\n![png](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_afb114327547.png)\n\n\n\n![png](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_009bd3feddac.png)\n\n\n\n![png](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_578023f11ca6.png)\n\n\n\n![png](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_002268999f5f.png)\n\n\n\n![png](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_1c07df87ec16.png)\n\n\n\n![png](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_a274c42b1bf7.png)\n\n\n\n![png](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_c36f474289a2.png)\n\n\n\n![png](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_7422dae392b0.png)\n\n\n\n![png](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_dd970788fa40.png)\n\n\n\n![png](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_6f41f25e6fa4.png)\n\n\n\n![png](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_7dc7331f8267.png)\n\n\n## Conclusion \n\nRecurrent Neural Networks are fabulous. They can learn to predict complex things. They can read multiple features from sequence data, and output variable length sequences of the same features, or of totally different features. Some people even use RNNs combined with other neural network architectures, such as CNNs, for automatic image captioning (CNN encoder for images, RNN decoder for the description). \n\nHere is what you learned:\n- Building a time series machine learning pipeline\n- Building a TensorFlow v2 encoder decoder sequence to sequence model\n- Building a clean machine learning pipeline using Neuraxle\n- Properly split the data for training and validation\n- Shuffling the data during training\n- Using minibatches to process the data using a MiniBatchSequentialPipeline\n\n## About\n\nConnect with me:\n- https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002F\n- https:\u002F\u002Fca.linkedin.com\u002Fin\u002Fchevalierg\n- https:\u002F\u002Ftwitter.com\u002Fguillaume_che\n\n## License & Citation\n\nThis project is free to use according to the [Apache 2.0 License](https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002Fseq2seq-signal-prediction\u002Fblob\u002Fmaster\u002FLICENSE) as long as you link to the project (citation), and that you respect the License (read the License for more details). You can cite by pointing to the following link: \n- https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002Fseq2seq-signal-prediction\n","# [用于时间序列预测的序列到序列（seq2seq）循环神经网络（RNN）](https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002Fseq2seq-signal-prediction)\n\n***注：您可以在此处找到配套的[seq2seq RNN 预测演示文稿幻灯片](https:\u002F\u002Fdrive.google.com\u002Fdrive\u002Ffolders\u002F1U0xQMxVespjQilMhYW4mDxN02IwEW67I)，以及用于运行本笔记本的 Google Colab 文件（如果您尚未在 Colab 中）。***\n\n这是一系列练习，您可以尝试解决这些练习来学习如何编写编码器-解码器结构的序列到序列循环神经网络（seq2seq RNN）。您可以通过解决不同的简单玩具信号预测问题来实践。Seq2seq 架构也可用于其他更复杂的应用场景，例如自然语言处理（NLP）。\n\n本项目提供了 4 个难度逐步递增的练习。我假定您至少对 RNN 的工作原理及其如何构建为最简单的编码器-解码器 seq2seq 模型（不带注意力机制）有一定的了解。如需深入了解 TensorFlow 中的 RNN，您可以访问我为此专门构建的[另一个 RNN 项目](https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002FLSTM-Human-Activity-Recognition)。\n\n当前项目最初是用法语编写的示例系列，但由于时间有限，我没有重新生成所有带有正确英文文本的图表。该项目最初是我为在魁北克 Web 大会（WAQ）上主讲的一场[大师班](https:\u002F\u002Fwebaquebec.org\u002Fclasses-de-maitre\u002Fdeep-learning-avec-tensorflow)第三小时的实践部分而设计的，该课程于 2017 年 3 月首次举办。\n\n## 如何使用此“.ipynb”Python 笔记本？\n\n我在[仓库](https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002Fseq2seq-signal-prediction)中提供了本教程的“.py”Python 版本，但直接在笔记本或 Google Colab 中运行代码更为方便。\n\n要运行笔记本，您可以在命令行中输入 `jupyter-notebook` 启动 Web 笔记本 IDE，并选择 `.ipynb` 文件。对于 Google Colab，如果您希望使用 GPU 运行代码，请确保依次点击“Runtime > Change Runtime Type”，并将“Python 3”的运行时类型设置为“GPU”。\n\n## 练习\n\n请注意，数据集会根据不同的练习而变化。大多数情况下，您需要调整神经网络的训练参数才能完成练习，但在某些阶段，则需要修改网络架构本身。本练习所使用的数据集位于 [`datasets.py`](https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002Fseq2seq-signal-prediction\u002Fblob\u002Fmaster\u002Fdatasets.py) 文件中。\n\n### 练习 1\n\n理论上，由于该练习中的信号是确定性的，因此可以实现完美的预测。神经网络的参数已设置为第一次训练时“勉强”可接受的值。您需要不断调整超参数，直到获得类似以下的预测结果：\n\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_8b46a99e555c.png\" \u002F>\n\n注意：神经网络仅能看到图表左侧的数据，并被训练用来预测右侧的内容（黄色部分为预测结果）。\n\n我们同时需要预测两条相互关联的时间序列。这意味着我们的神经网络处理的是多维数据。一个简单的例子就是，将多个股票代码的历史价格作为输入，然后利用神经网络预测这些股票未来的同步走势。这正是我们在练习 4 中将要做的——同时预测比特币兑美元和欧元的价格。\n\n### 练习 2\n\n与前一个练习不同，这里为了简化起见，我们只需预测一条信号。然而，这条信号是由两个波长和偏移量不断变化的正弦波叠加而成的，并且其波长被限制在特定的最小值和最大值范围内。\n\n要顺利完成此练习，您需要调整神经网络的超参数。我们建议先尝试以下超参数组合：\n\n- `n_samples = 125000`\n- `epochs = 1`\n- `batch_size = 50`\n- `hidden_dim = 35`\n\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_b21932ed5500.png\" \u002F>\n\n以下是使用更大型神经网络（包含 3 个堆叠的循环单元，每个单元有 500 个隐藏单元）得到的预测结果：\n\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_f7ca6ac96ad2.png\" \u002F>\n\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_7db3277e746a.png\" \u002F>\n\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_7db3277e746a.png\" \u002F>\n\n需要注意的是，如果采用更小规模的神经网络，配合更好的训练超参数、更长时间的训练、添加 Dropout 等技术手段，同样可以获得更优的结果。\n\n### 练习 3\n\n本练习与前一个类似，唯一的区别在于输入编码器的数据是带有噪声的。而期望的输出则没有噪声，这使得任务难度有所增加。在这种特定的数据背景下，我们可以将我们的神经网络称为去噪自回归自动编码器。以下是一个训练样本（以及预测结果）的示例：\n\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_854639a1653f.png\" \u002F>\n\n因此，神经网络的任务是去除信号中的噪声，从而准确地预测其未来的平滑值。以下是针对该数据集版本的一些更优预测示例：\n\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_60d1f51cb9b8.png\" \u002F>\n\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_3400da8e93f5.png\" \u002F>\n\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_02d9ae19b8e1.png\" \u002F>\n\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_d1075e4bc0c0.png\" \u002F>\n\n正如我在练习 2 中提到的那样，此处同样有可能获得更好的结果。此外，我们也可以要求您从噪声输入中重建去噪后的信号（而不是预测其未来值），即将其作为去噪自动编码器来使用。这种架构还适用于数据压缩，例如图像处理等场景。\n\n### 练习 4\n\n本练习比之前的要困难得多，更像是一个开放性的建议。任务是预测比特币价格的未来值。我们这里有一些比特币每日市场数据，即 BTC\u002FUSD 和 BTC\u002FEUR 的汇率。仅凭这些数据还不足以构建一个优秀的预测模型——至少需要分钟级甚至秒级的精确数据，这样才会更有意义。以下是一个基于实际未来值的预测结果：该神经网络并未在这些未来值上进行训练，因此在拥有足够好的、针对该任务训练过的模型的情况下，这仍是一个合法的预测：\n\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_afe85f096617.png\" \u002F>\n\n免责声明：这个对未来值的预测确实非常出色，但请不要期望使用如此少量的数据时，预测效果总是这么好（顺便提一句：本项目中的其他预测图表大多属于“一般”水平，唯有这一张例外）。我并没有花太多时间将该模型与其他金融模型进行比较。对于本练习，你可以尝试向模型中引入更多有价值的金融数据，以提高预测的准确性。提醒一下，我在 [`datasets.py`](https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002Fseq2seq-signal-prediction\u002Fblob\u002Fmaster\u002Fdatasets.py) 中提供了数据集的代码，但你可以用更全面的数据来替换，从而更准确地预测比特币价格。\n\n该模型的输入和输出维度均为二维，分别对应 BTC\u002FUSD 和 BTC\u002FEUR。例如，你还可以创建额外的输入维度或数据流，比如天气数据以及更多的金融指标，如标普 500 指数、道琼斯工业平均指数等。更具创意的输入数据还可以是正弦波（或其他形状的波形，如锯齿波、三角波，或者同时使用 `cos` 和 `sin` 信号），用来表示分钟、小时、天、周、月、年、月相等不同时间尺度上的波动变化（正如我们在 [Neuraxio 时间序列解决方案](https:\u002F\u002Fwww.neuraxio.com\u002Fen\u002Ftime-series-solution) 中所做的那样）。此外，还可以结合社交媒体上关于“比特币”一词的情感分析流，作为另一路更加以人为本且抽象的输入信号。另外，了解“比特币在全球哪些地区使用最为广泛”也很有意思，相关信息可以参考：[bitcoin heatmap world](http:\u002F\u002Fimages.google.com\u002Fsearch?tbm=isch&q=bitcoin+heatmap+world)。\n\n综合以上提到的各种示例，我们可以将所有这些内容作为每一步的时间序列输入特征：(BTC\u002FUSD, BTC\u002FEUR, 道琼斯指数, 标普 500 指数, 当前小时数, 星期几, 本月第几天, 一年中的第几周, 年份, 月相, 美国天气数据, 欧洲天气数据, 社交媒体情感分析)。最终，输出可以是两维或多维：(BTC\u002FUSD, BTC\u002FEUR)。\n\n这种预测概念以及类似的时间序列预测算法，可以应用于许多领域，例如工业 4.0 中的自动纠错设备、生产链中的质量保证、交通流量预测、天气预报、运动与行为预测，以及其他各种短期和中期的统计预测或预报任务。\n\n## 安装依赖\n\n\n```python\n!pip install tensorflow-gpu==2.1 neuraxle==0.3.1 neuraxle_tensorflow==0.1.0\n```\n\n需求已满足：tensorflow-gpu==2.1，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (2.1.0)\n    需求已满足：neuraxle==0.3.1，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (0.3.1)\n    需求已满足：neuraxle_tensorflow==0.1.0，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages (0.1.0)\n    需求已满足：google-pasta>=0.1.6，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 tensorflow-gpu==2.1）(0.1.8)\n    需求已满足：opt-einsum>=2.3.2，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 tensorflow-gpu==2.1）(3.1.0)\n    需求已满足：tensorboard\u003C2.2.0,>=2.1.0，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 tensorflow-gpu==2.1）(2.1.0)\n    需求已满足：gast==0.2.2，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 tensorflow-gpu==2.1）(0.2.2)\n    需求已满足：protobuf>=3.8.0，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 tensorflow-gpu==2.1）(3.10.0)\n    需求已满足：termcolor>=1.1.0，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 tensorflow-gpu==2.1）(1.1.0)\n    需求已满足：wrapt>=1.11.1，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 tensorflow-gpu==2.1）(1.11.2)\n    需求已满足：absl-py>=0.7.0，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 tensorflow-gpu==2.1）(0.9.0)\n    需求已满足：six>=1.12.0，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 tensorflow-gpu==2.1）(1.12.0)\n    需求已满足：keras-applications>=1.0.8，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 tensorflow-gpu==2.1）(1.0.8)\n    需求已满足：tensorflow-estimator\u003C2.2.0,>=2.1.0rc0，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 tensorflow-gpu==2.1）(2.1.0)\n    需求已满足：astor>=0.6.0，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 tensorflow-gpu==2.1）(0.8.1)\n    需求已满足：keras-preprocessing>=1.1.0，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 tensorflow-gpu==2.1）(1.1.0)\n    需求已满足：numpy\u003C2.0,>=1.16.0，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 tensorflow-gpu==2.1）(1.17.5)\n    需求已满足：wheel>=0.26；python_version >= \"3\"，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 tensorflow-gpu==2.1）(0.33.6)\n    需求已满足：grpcio>=1.8.6，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 tensorflow-gpu==2.1）(1.15.0)\n    需求已满足：scipy==1.4.1；python_version >= \"3\"，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 tensorflow-gpu==2.1）(1.4.1)\n    需求已满足：matplotlib，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 neuraxle==0.3.1）(3.1.2)\n    需求已满足：Flask-RESTful>=0.3.7，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 neuraxle==0.3.1）(0.3.7)\n    需求已满足：conv==0.2，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 neuraxle==0.3.1）(0.2)\n    需求已满足：Flask>=1.1.1，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 neuraxle==0.3.1）(1.1.1)\n    需求已满足：joblib>=0.13.2，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 neuraxle==0.3.1）(0.14.1)\n    需求已满足：scikit-learn>=0.20.3，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 neuraxle==0.3.1）(0.22.1)\n    需求已满足：werkzeug>=0.11.15，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1）(0.16.0)\n    需求已满足：google-auth\u003C2,>=1.6.3，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1）(1.10.1)\n    需求已满足：google-auth-oauthlib\u003C0.5,>=0.4.1，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1）(0.4.1)\n    需求已满足：markdown>=2.6.8，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1）(3.1.1)\n    需求已满足：setuptools>=41.0.0，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1）(42.0.2)\n    需求已满足：requests\u003C3,>=2.21.0，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1）(2.21.0)\n    需求已满足：h5py，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 keras-applications>=1.0.8->tensorflow-gpu==2.1）(2.8.0)\n    需求已满足：kiwisolver>=1.0.1，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 matplotlib->neuraxle==0.3.1）(1.1.0)\n    需求已满足：pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 matplotlib->neuraxle==0.3.1）(2.4.6)\n    需求已满足：python-dateutil>=2.1，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 matplotlib->neuraxle==0.3.1）(2.6.1)\n    需求已满足：cycler>=0.10，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 matplotlib->neuraxle==0.3.1）(0.10.0)\n    需求已满足：aniso8601>=0.82，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 Flask-RESTful>=0.3.7->neuraxle==0.3.1）(8.0.0)\n    需求已满足：pytz，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 Flask-RESTful>=0.3.7->neuraxle==0.3.1）(2018.9)\n    需求已满足：click>=5.1，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 Flask>=1.1.1->neuraxle==0.3.1）(7.0)\n    需求已满足：Jinja2>=2.10.1，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 Flask>=1.1.1->neuraxle==0.3.1）(2.10.3)\n    需求已满足：itsdangerous>=0.24，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 Flask>=1.1.1->neuraxle==0.3.1）(1.1.0)\n    需求已满足：rsa\u003C4.1,>=3.1.4，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 google-auth\u003C2,>=1.6.3->tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1）(4.0)\n    需求已满足：pyasn1-modules>=0.2.1，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 google-auth\u003C2,>=1.6.3->tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1）(0.2.7)\n    需求已满足：cachetools\u003C5.0,>=2.0.0，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 google-auth\u003C2,>=1.6.3->tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1）(4.0.0)\n    需求已满足：requests-oauthlib>=0.7.0，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 google-auth-oauthlib\u003C0.5,>=0.4.1->tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1）(1.3.0)\n    需求已满足：chardet\u003C3.1.0,>=3.0.2，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 requests\u003C3,>=2.21.0->tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1）(3.0.4)\n    需求已满足：urllib3\u003C1.25,>=1.21.1，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 requests\u003C3,>=2.21.0->tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1）(1.24.3)\n    需求已满足：certifi>=2017.4.17，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 requests\u003C3,>=2.21.0->tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1）(2019.11.28)\n    需求已满足：idna\u003C2.9,>=2.5，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 requests\u003C3,>=2.21.0->tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1）(2.8)\n    需求已满足：MarkupSafe>=0.23，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 Jinja2>=2.10.1->Flask>=1.1.1->neuraxle==0.3.1）(1.1.1)\n    需求已满足：pyasn1>=0.1.3，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 rsa\u003C4.1,>=3.1.4->google-auth\u003C2,>=1.6.3->tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1）(0.4.8)\n    需求已满足：oauthlib>=3.0.0，位于 \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages（源自 requests-oauthlib>=0.7.0->google-auth-oauthlib\u003C0.5,>=0.4.1->tensorboard\u003C2.2.0,>=2.1.0->tensorflow-gpu==2.1）(3.1.0)\n\n```python\nimport urllib\n\ndef download_import(filename):\n    with open(filename, \"wb\") as f:\n        # 之所以需要以这种方式下载，是因为 Colab 是从一个仅“与您共享”的 Google Drive 文件夹中运行的。\n        # https:\u002F\u002Fdrive.google.com\u002Fdrive\u002Ffolders\u002F1U0xQMxVespjQilMhYW4mDxN02IwEW67I\n        url = 'https:\u002F\u002Fraw.githubusercontent.com\u002Fguillaume-chevalier\u002Fseq2seq-signal-prediction\u002Fmaster\u002F{}'.format(filename)\n        f.write(urllib.request.urlopen(url).read())\n\ndownload_import(\"datasets.py\")\ndownload_import(\"plotting.py\")\ndownload_import(\"steps.py\")\n```\n\n\n```python\nfrom typing import List\nfrom logging import warning\n\nimport tensorflow as tf\nfrom neuraxle.data_container import DataContainer\nfrom neuraxle.hyperparams.space import HyperparameterSamples\nfrom neuraxle.metaopt.random import ValidationSplitWrapper\nfrom neuraxle.metrics import MetricsWrapper\nfrom neuraxle.pipeline import Pipeline, MiniBatchSequentialPipeline\nfrom neuraxle.steps.data import EpochRepeater, DataShuffler\nfrom neuraxle.steps.flow import TrainOnlyWrapper\nfrom neuraxle.steps.loop import ForEachDataInput\nfrom sklearn.metrics import mean_squared_error\nfrom tensorflow_core.python.client import device_lib\nfrom tensorflow_core.python.keras import Input, Model\nfrom tensorflow_core.python.keras.layers import GRUCell, RNN, Dense\nfrom tensorflow_core.python.training.adam import AdamOptimizer\n\nfrom datasets import generate_data\nfrom datasets import metric_3d_to_2d_wrapper\nfrom neuraxle_tensorflow.tensorflow_v1 import TensorflowV1ModelStep\nfrom neuraxle_tensorflow.tensorflow_v2 import Tensorflow2ModelStep\nfrom plotting import plot_metrics\nfrom steps import MeanStdNormalizer, ToNumpy, PlotPredictionsWrapper\n\n%matplotlib inline\n```\n\n\n```python\ndef choose_tf_device():\n    \"\"\"\n    选择 TensorFlow 设备（例如：如果有 GPU 则使用 GPU）进行计算。\n    \"\"\"\n    tf.debugging.set_log_device_placement(True)\n    devices = [x.name for x in device_lib.list_local_devices()]\n    print('您可以使用的 TensorFlow 设备有：{}'.format(devices))\n    try:\n        chosen_device = [d for d in devices if 'gpu' in d.lower()][0]\n    except:\n        warning(\n            \"未找到 GPU 设备。请确保在 `Runtime > Change Runtime Type` 中选择 Python 3 的 GPU 配置。\")\n        chosen_device = devices[0]\n    print('选定设备：{}'.format(chosen_device))\n    return chosen_device\n\nchosen_device = choose_tf_device()\n```\n\n    您可以使用的 TensorFlow 设备有： ['\u002Fdevice:CPU:0', '\u002Fdevice:XLA_CPU:0', '\u002Fdevice:XLA_GPU:0', '\u002Fdevice:GPU:0']\n    选定设备： \u002Fdevice:XLA_GPU:0\n\n\n\n\n\n## 神经网络架构定义\n\n### 基本的序列到序列（seq2seq）RNN\n\n以下是一个基本的序列到序列神经网络架构。“ABC”是过去的输入，“WXYZ”既是未来的输出，也是作为反馈回路的未来输入。这种反馈回路在某些情况下已被证明可以提升 RNN 的性能（[了解更多](https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002FAwesome-Deep-Learning-Resources#recurrent-neural-networks)）。\n\n\u003Cimg src=\"https:\u002F\u002Fwww.tensorflow.org\u002Fimages\u002Fbasic_seq2seq.png\" \u002F>\n\n但在我们的案例中，我们不会采用这样的反馈回路，因为这需要在训练和测试过程中进行更复杂的采样，对于今天的实践示例来说会过于复杂。\n\n### 我们的堆叠 GRU 序列到序列 RNN\n\n我们采用的是如下方法：“H”是编码器 RNN 最后一个时间步的隐藏状态输出。我们将这个值在未来的各个时间步上复制，作为 RNN 的未来输入，从而使它在预测未来时始终能够记住当前的上下文信息。\n\n\u003Cimg src=\"https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_fde9af34637f.png\" \u002F>\n\n需要注意的是，我们也可以在此处引入注意力机制。这样做的好处是，神经网络可以在未来的每一个时间步重新分析过去的信息，如果需要的话。注意力机制在机器翻译（MT）等场景中更为有用，因为在这些场景中，有时需要逐字逐句地回顾之前的内容，而不是仅仅依赖于一次性读取所有内容后形成的短期记忆。近年来，像 BERT 这样的机器翻译方法（[关于 BERT 的更多信息](https:\u002F\u002Fwww.umaneo.com\u002Fpost\u002Fa-review-of-recent-natural-language-processing-approaches) \u002F [BERT 使用示例](https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002FReuBERT)）已经完全基于注意力机制，而不再使用 RNN，尽管这样做也存在一些权衡。\n\n## 创建 TensorFlow 2 模型\n\n让我们继续编写上方图片中所示的内容。\n\n\n\n```python\ndef create_model(step: Tensorflow2ModelStep) -> tf.keras.Model:\n    \"\"\"\n   创建一个 TensorFlow v2 序列到序列（seq2seq）编码器-解码器模型。\n\n   :param step: TensorFlow v2 的 Neuraxle 基础步骤（Tensorflow2ModelStep）\n   :return: TensorFlow v2 Keras 模型\n    \"\"\"\n    # 形状：(batch_size, seq_length, input_dim)\n    encoder_inputs = Input(\n        shape=(None, step.hyperparams['input_dim']),\n        batch_size=None,\n        dtype=tf.dtypes.float32,\n        name='encoder_inputs'\n    )\n\n    last_encoder_outputs, last_encoders_states = _create_encoder(step, encoder_inputs)\n    decoder_outputs = _create_decoder(step, last_encoder_outputs, last_encoders_states)\n\n    return Model(encoder_inputs, decoder_outputs)\n\ndef _create_encoder(step: Tensorflow2ModelStep, encoder_inputs: Input) -> (tf.Tensor, List[tf.Tensor]):\n    \"\"\"\n   使用 GRU 单元创建编码器 RNN。GRU 单元与 LSTM 单元类似。\n\n   :param step: TensorFlow v2 的 Neuraxle 基础步骤（类 Tensorflow2ModelStep）\n    :param encoder_inputs: 编码器输入层，形状为 (batch_size, seq_length, input_dim)\n    :return: (最后的编码器输出, 最后堆叠的编码器状态)\n                last_encoder_outputs 形状：(batch_size, hidden_dim)\n                last_encoder_states 形状：(layers_stacked_count, batch_size, hidden_dim)\n    \"\"\"\n    encoder = RNN(cell=_create_stacked_rnn_cells(step), return_sequences=False, return_state=True)\n\n    last_encoder_outputs_and_states = encoder(encoder_inputs)\n    # last_encoder_outputs 形状：(batch_size, hidden_dim)\n    # last_encoder_states 形状：(layers_stacked_count, batch_size, hidden_dim)\n\n    # 参考：https:\u002F\u002Fwww.tensorflow.org\u002Fapi_docs\u002Fpython\u002Ftf\u002Fkeras\u002Flayers\u002FRNN?version=stable#output_shape_2\n    last_encoder_outputs, *last_encoders_states = last_encoder_outputs_and_states\n    return last_encoder_outputs, last_encoders_states\n\ndef _create_decoder(step: Tensorflow2ModelStep, last_encoder_outputs: tf.Tensor, last_encoders_states: List[tf.Tensor]) -> tf.Tensor:\n    \"\"\"\n   使用 GRU 单元创建解码器 RNN。\n\n   :param step: TensorFlow v2 的 Neuraxle 基础步骤（Tensorflow2ModelStep）\n    :param last_encoders_states: 最后的编码器状态张量\n    :param last_encoder_outputs: 最后的编码器输出张量\n    :return: 解码器输出\n    \"\"\"\n    decoder_lstm = RNN(cell=_create_stacked_rnn_cells(step), return_sequences=True, return_state=False)\n\n    last_encoder_output = tf.expand_dims(last_encoder_outputs, axis=1)\n    # 最后的编码器输出形状：(batch_size, 1, hidden_dim)\n\n    replicated_last_encoder_output = tf.repeat(\n        input=last_encoder_output,\n        repeats=step.hyperparams['window_size_future'],\n        axis=1\n    )\n    # 复制后的最后编码器输出形状：(batch_size, window_size_future, hidden_dim)\n\n    decoder_outputs = decoder_lstm(replicated_last_encoder_output, initial_state=last_encoders_states)\n    # 解码器输出形状：(batch_size, window_size_future, hidden_dim)\n\n    decoder_dense = Dense(step.hyperparams['output_dim'])\n    # 解码器输出形状：(batch_size, window_size_future, output_dim)\n\n    return decoder_dense(decoder_outputs)\n\ndef _create_stacked_rnn_cells(step: Tensorflow2ModelStep) -> List[GRUCell]:\n    \"\"\"\n   创建 `layers_stacked_count` 个 GRU 单元，并将它们堆叠在一起。\n   它们的神经元层数为 `hidden_dim`。\n\n   :param step: TensorFlow v2 的 Neuraxle 基础步骤（Tensorflow2ModelStep）\n    :return: GRU 单元列表\n    \"\"\"\n    cells = []\n    for _ in range(step.hyperparams['layers_stacked_count']):\n        cells.append(GRUCell(step.hyperparams['hidden_dim']))\n\n    return cells\n```\n\n## 创建损失函数\n\n使用均方误差（MSE）和权重衰减（L2 正则化）。\n\n\n```python\ndef create_loss(step: Tensorflow2ModelStep, expected_outputs: tf.Tensor, predicted_outputs: tf.Tensor) -> tf.Tensor:\n    \"\"\"\n    创建模型损失。\n\n   :param step: TensorFlow v2 的 Neuraxle 基础步骤（Tensorflow2ModelStep）\n   :param expected_outputs: 预期输出，形状为 (batch_size, window_size_future, output_dim)\n   :param predicted_outputs: 预测输出，形状为 (batch_size, window_size_future, output_dim)\n   :return: 损失（一个浮点类型的 tf 张量）\n    \"\"\"\n    l2 = step.hyperparams['lambda_loss_amount'] * sum(\n        tf.reduce_mean(tf.nn.l2_loss(tf_var))\n        for tf_var in step.model.trainable_variables\n    )\n\n    output_loss = sum(\n        tf.reduce_mean(tf.nn.l2_loss(pred - expected))\n        for pred, expected in zip(predicted_outputs, expected_outputs)\n    ) \u002F float(len(predicted_outputs))\n\n    return output_loss + l2\n```\n\n## 创建优化器\n\nAdam 通常表现优异。\n\n\n```python\ndef create_optimizer(step: TensorflowV1ModelStep) -> AdamOptimizer:\n    \"\"\"\n   创建 TensorFlow 2 的优化器：这里是 AdamOptimizer。\n\n   :param step: TensorFlow v2 的 Neuraxle 基础步骤（Tensorflow2ModelStep）\n    :return: 优化器\n    \"\"\"\n    return AdamOptimizer(learning_rate=step.hyperparams['learning_rate'])\n```\n\n## 生成或加载数据\n\n要更改您正在进行的练习，请更改 `exercise_number` 变量的值（即下方代码单元的第一行）：\n\n\n```python\nexercice_number = 1\nprint('exercice {}\\n=================='.format(exercice_number))\n\ndata_inputs, expected_outputs = generate_data(\n    # 参见：https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002Fseq2seq-signal-prediction\u002Fblob\u002Fmaster\u002Fdatasets.py\n    exercice_number=exercice_number,\n    n_samples=None,\n    window_size_past=None,\n    window_size_future=None\n)\n\nprint('data_inputs shape: {} => (n_samples, window_size_past, input_dim)'.format(data_inputs.shape))\nprint('expected_outputs shape: {} => (n_samples, window_size_future, output_dim)'.format(expected_outputs.shape))\n\nsequence_length = data_inputs.shape[1]\ninput_dim = data_inputs.shape[2]\noutput_dim = expected_outputs.shape[2]\n\nbatch_size = 100\nepochs = 15\nvalidation_size = 0.15\nmax_plotted_validation_predictions = 10\n```\n\n    练习 1\n    ==================\n    data_inputs 形状：(1000, 10, 2) => (n_samples, window_size_past, input_dim)\n    expected_outputs 形状：(1000, 10, 2) => (n_samples, window_size_future, output_dim)\n\n\n## 神经网络的超参数\n\n\n```python\nseq2seq_pipeline_hyperparams = HyperparameterSamples({\n    'hidden_dim': 12,\n    'layers_stacked_count': 2,\n    'lambda_loss_amount': 0.0003,\n    'learning_rate': 0.001,\n    'window_size_future': sequence_length,\n    'output_dim': output_dim,\n    'input_dim': input_dim\n})\n\nprint('hyperparams: {}'.format(seq2seq_pipeline_hyperparams))\n```\n\n    超参数：HyperparameterSamples([('hidden_dim', 12), ('layers_stacked_count', 2), ('lambda_loss_amount', 0.0003), ('learning_rate', 0.001), ('window_size_future', 10), ('output_dim', 2), ('input_dim', 2)])\n\n## 管道\n\n看到[脏乱的机器学习代码](https:\u002F\u002Fwww.neuraxio.com\u002Fen\u002Fblog\u002Fclean-code\u002F2019\u002F12\u002F26\u002Fmachine-learning-competition-code.html)几乎已经成为行业常态。而这无疑也是导致[87%的数据科学项目从未投入生产](https:\u002F\u002Fventurebeat.com\u002F2019\u002F07\u002F19\u002Fwhy-do-87-of-data-science-projects-never-make-it-into-production\u002F)的原因之一。\n\n在这里，我们使用先进的设计模式（管道与过滤器）来实现所谓的[整洁机器学习](https:\u002F\u002Fwww.neuraxio.com\u002Fen\u002Fblog\u002Fneuraxle\u002F2019\u002F10\u002F26\u002Fneat-machine-learning-pipelines.html)。这些设计模式的灵感来源于[scikit-learn的Pipeline类](https:\u002F\u002Fwww.neuraxio.com\u002Fen\u002Fblog\u002Fscikit-learn\u002F2020\u002F01\u002F03\u002Fwhat-is-wrong-with-scikit-learn.html)。\n\n### 定义深度学习管道\n\n首先，我们使用[TF2ModelStep](https:\u002F\u002Fgithub.com\u002FNeuraxio\u002FNeuraxle-TensorFlow)来定义管道。MeanStdNormalizer可以帮助我们对数据进行归一化，因为神经网络需要处理归一化的输入数据。\n\n\n\n```python\nfeature_0_metric = metric_3d_to_2d_wrapper(mean_squared_error)\nmetrics = {'mse': feature_0_metric}\n\nsignal_prediction_pipeline = Pipeline([\n    ForEachDataInput(MeanStdNormalizer()),\n    ToNumpy(),\n    PlotPredictionsWrapper(Tensorflow2ModelStep(\n        # 参见：https:\u002F\u002Fgithub.com\u002FNeuraxio\u002FNeuraxle-TensorFlow\n        create_model=create_model,\n        create_loss=create_loss,\n        create_optimizer=create_optimizer,\n        expected_outputs_dtype=tf.dtypes.float32,\n        data_inputs_dtype=tf.dtypes.float32,\n        print_loss=False,\n        device_name=chosen_device\n).set_hyperparams(seq2seq_pipeline_hyperparams))]).set_name('SignalPrediction')\n\n```\n\n### 定义如何训练我们的深度学习管道\n\n最后，我们将管道包裹在EpochRepeater、ValidationSplitWrapper、DataShuffler、MiniBatchSequentialPipeline和MetricsWrapper中，以处理训练所需的所有步骤。有关这些组件的更多信息，请参阅[Neuraxle的文档](https:\u002F\u002Fwww.neuraxle.org\u002Fstable\u002Findex.html)。\n\n\n```python\n\npipeline = Pipeline([EpochRepeater(\n    ValidationSplitWrapper(\n        MetricsWrapper(Pipeline([\n            TrainOnlyWrapper(DataShuffler()),\n            MiniBatchSequentialPipeline([\n                MetricsWrapper(\n                    signal_prediction_pipeline,\n                    metrics=metrics,\n                    name='batch_metrics'\n                )], batch_size=batch_size)\n            ]), \n            metrics=metrics,\n            name='epoch_metrics',\n            print_metrics=True\n        ),\n        test_size=validation_size,\n        scoring_function=feature_0_metric), \n    epochs=epochs)\n])\n\n```\n\n    \u002Fusr\u002Flocal\u002Flib\u002Fpython3.6\u002Fdist-packages\u002Fneuraxle\u002Fpipeline.py:353: UserWarning: 正在用MiniBatchSequentialPipeline.batch_size替换MiniBatchSequentialPipeline[Joiner].batch_size。\n      'Replacing {}[{}].batch_size by {}.batch_size.'.format(self.name, step.name, self.name))\n\n\n## 训练神经网络\n\n现在是时候将模型拟合到数据上了。\n\n\n\n```python\n\npipeline, outputs = pipeline.fit_transform(data_inputs, expected_outputs)\n\n```\n\n在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 RandomUniform 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 Sub 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 Mul 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 Add 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 VarHandleOp 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 VarIsInitializedOp 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 LogicalNot 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 Assert 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 AssignVariableOp 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 RandomStandardNormal 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 Qr 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 DiagPart 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 Sign 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 Transpose 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 Reshape 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 VarHandleOp 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 Fill 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 Cast 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference_keras_scratch_graph_365 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference_keras_scratch_graph_370 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference_keras_scratch_graph_375 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference_keras_scratch_graph_380 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference_keras_scratch_graph_385 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference_keras_scratch_graph_390 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference_keras_scratch_graph_395 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference_keras_scratch_graph_400 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference_keras_scratch_graph_405 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference_keras_scratch_graph_410 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference_keras_scratch_graph_415 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 VarHandleOp 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 VarHandleOp 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 VarHandleOp 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 AssignVariableOp 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 Shape 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 StridedSlice 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 Unpack 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 StridedSlice 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 ReadVariableOp 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 Unpack 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 MatMul 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 BiasAdd 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 Split 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 SplitV 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 AddV2 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 Sigmoid 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 Tanh 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 Less 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 AddV2 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 Pack 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __forward__defun_call_1537 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __forward__defun_call_1547 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference__defun_call_1557 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference__defun_call_1566 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __forward__defun_call_1580 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __forward__defun_call_1606 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference__defun_call_1617 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __forward__defun_call_1631 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 L2Loss 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 Mean 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 RealDiv 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 BroadcastGradientArgs 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 Sum 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 Neg 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 Tile 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 StridedSliceGrad 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 AddN 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 BiasAddGrad 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 MatMul 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 MatMul 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 InvertPermutation 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 TanhGrad 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 AddN 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 SigmoidGrad 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 ConcatV2 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 Pack 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 AddN 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference___backward__defun_call_1625_1632 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference___backward__defun_call_1589_1607 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference___backward__defun_call_1574_1581 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference___backward__defun_call_1531_1538 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 AddN 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 VarHandleOp 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 ResourceApplyAdam 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference__defun_call_1530 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference__defun_call_1544 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference__defun_call_1573 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference__defun_call_1588 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference__defun_call_1624 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __forward__defun_call_65540 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __forward__defun_call_65550 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __forward__defun_call_65567 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __forward__defun_call_65593 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __forward__defun_call_65610 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 AddN 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference___backward__defun_call_65604_65611 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference___backward__defun_call_65576_65594 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference___backward__defun_call_65561_65568 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference___backward__defun_call_65534_65541 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference__defun_call_65533 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference__defun_call_65547 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference__defun_call_65560 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference__defun_call_65575 操作\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 __inference__defun_call_65603 操作\n    {'mse': 0.18414642847122925}\n    {'mse': 0.1778781379709343}\n    {'mse': 0.1723181842129053}\n    {'mse': 0.1658200688421554}\n    {'mse': 0.1591329577983185}\n    {'mse': 0.15131258011101834}\n    {'mse': 0.1436201535512516}\n    {'mse': 0.1343595503512161}\n    {'mse': 0.12474072112690562}\n    {'mse': 0.11462532630747631}\n    {'mse': 0.10271182130173581}\n    {'mse': 0.0906442166022616}\n    {'mse': 0.07585859336447773}\n    {'mse': 0.06317439259405164}\n    {'mse': 0.04988300184267241}\n    {'mse': 0.041345448752856694}\n    {'mse': 0.034553488508200454}\n    {'mse': 0.03218617403485365}\n    {'mse': 0.02922688138678744}\n    {'mse': 0.02631547230588055}\n    {'mse': 0.022075968214915552}\n    {'mse': 0.018800000904722468}\n    {'mse': 0.01640079469351695}\n    {'mse': 0.014737265865397323}\n    {'mse': 0.013079363146911618}\n    {'mse': 0.01166897820815228}\n    {'mse': 0.010537850442431971}\n    {'mse': 0.00938083864872879}\n    {'mse': 0.008495135058422493}\n    {'mse': 0.007566329717239811}\n\n## 可视化测试预测\n\n查看您的训练效果。\n\n\n```python\nplot_metrics(pipeline=pipeline, exercice_number=exercice_number)\n```\n\n    最后一次训练集 MSE：0.008495135058422493\n    最佳训练集 MSE：0.008495135058422493\n    最后一次验证集 MSE：0.007566329717239811\n    最佳验证集 MSE：0.007566329717239811\n\n\n\n![png](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_2df3686cab3f.png)\n\n\n\n```python\ndef plot_predictions(data_inputs, expected_outputs, pipeline, max_plotted_predictions):\n    _, _, data_inputs_validation, expected_outputs_validation = \\\n        pipeline.get_step_by_name('ValidationSplitWrapper').split(data_inputs, expected_outputs)\n\n    pipeline.apply('toggle_plotting')\n    pipeline.apply('set_max_plotted_predictions', max_plotted_predictions)\n\n    signal_prediction_pipeline = pipeline.get_step_by_name('SignalPrediction')\n    signal_prediction_pipeline.transform_data_container(DataContainer(\n        data_inputs=data_inputs_validation,\n        expected_outputs=expected_outputs_validation\n    ))\n\nplot_predictions(data_inputs, expected_outputs, pipeline, max_plotted_validation_predictions)\n```\n\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 op __inference__defun_call_1562783\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 op __inference__defun_call_1562789\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 op __inference__defun_call_1562798\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 op __inference__defun_call_1562805\n    在设备 \u002Fjob:localhost\u002Freplica:0\u002Ftask:0\u002Fdevice:XLA_GPU:0 上执行 op __inference__defun_call_1562813\n\n\n\n![png](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_afb114327547.png)\n\n\n\n![png](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_009bd3feddac.png)\n\n\n\n![png](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_578023f11ca6.png)\n\n\n\n![png](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_002268999f5f.png)\n\n\n\n![png](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_1c07df87ec16.png)\n\n\n\n![png](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_a274c42b1bf7.png)\n\n\n\n![png](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_c36f474289a2.png)\n\n\n\n![png](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_7422dae392b0.png)\n\n\n\n![png](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_dd970788fa40.png)\n\n\n\n![png](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_6f41f25e6fa4.png)\n\n\n\n![png](https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_readme_7dc7331f8267.png)\n\n\n## 结论\n\n循环神经网络非常出色。它们能够学习预测复杂的数据模式。它们可以从序列数据中读取多个特征，并输出长度可变的序列，这些序列可以是相同特征的，也可以是完全不同特征的。有些人甚至将 RNN 与其他神经网络架构（如 CNN）结合使用，用于自动图像描述生成任务（用 CNN 对图像进行编码，再用 RNN 对描述进行解码）。\n\n您在本项目中学到了：\n- 如何构建时间序列机器学习流水线\n- 如何构建 TensorFlow v2 的编码器-解码器序列到序列模型\n- 如何使用 Neuraxle 构建整洁的机器学习流水线\n- 如何正确地将数据划分为训练集和验证集\n- 在训练过程中对数据进行打乱\n- 使用小批量数据处理技术，通过 MiniBatchSequentialPipeline 处理数据\n\n## 关于\n\n与我联系：\n- https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002F\n- https:\u002F\u002Fca.linkedin.com\u002Fin\u002Fchevalierg\n- https:\u002F\u002Ftwitter.com\u002Fguillaume_che\n\n## 许可与引用\n\n本项目可根据 [Apache 2.0 许可协议](https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002Fseq2seq-signal-prediction\u002Fblob\u002Fmaster\u002FLICENSE) 自由使用，但前提是您需要链接到该项目（即引用），并且遵守该许可协议（详情请参阅许可协议）。您可以引用以下链接：\n- https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002Fseq2seq-signal-prediction","# seq2seq-signal-prediction 快速上手指南\n\n本项目提供了一系列练习，旨在帮助开发者学习如何编写用于时间序列预测的编码器 - 解码器（Encoder-Decoder）序列到序列循环神经网络（seq2seq RNN）。通过完成从简单到复杂的四个练习，你将掌握如何处理多维信号、去噪以及进行金融数据预测。\n\n## 环境准备\n\n*   **系统要求**：支持 Python 3.6+ 的操作系统（Linux, macOS, Windows）。\n*   **运行环境**：推荐使用 **Jupyter Notebook** 或 **Google Colab** 运行 `.ipynb` 文件，以获得最佳的交互式体验。\n    *   若在 Google Colab 中运行以利用 GPU 加速，请务必在菜单栏选择 `Runtime (运行时)` > `Change Runtime Type (更改运行时类型)`，并将 `Hardware Accelerator (硬件加速器)` 设置为 `GPU`。\n*   **前置知识**：建议具备基础的 RNN 工作原理知识，了解基本的 Encoder-Decoder 架构。\n\n## 安装步骤\n\n本项目依赖 TensorFlow 2.1 及 Neuraxle 库。你可以直接在命令行或 Notebook 单元格中执行以下命令进行安装。\n\n> **提示**：国内用户若遇到下载速度慢的问题，可添加 `-i https:\u002F\u002Fpypi.tuna.tsinghua.edu.cn\u002Fsimple` 参数使用清华镜像源加速安装。\n\n```bash\npip install tensorflow-gpu==2.1 neuraxle==0.3.1 neuraxle_tensorflow==0.1.0\n```\n\n如果在 Jupyter Notebook 或 Google Colab 中运行，请使用：\n\n```python\n!pip install tensorflow-gpu==2.1 neuraxle==0.3.1 neuraxle_tensorflow==0.1.0\n```\n\n## 基本使用\n\n本项目核心代码位于 `.ipynb` Notebook 文件中。启动方式如下：\n\n1.  **本地运行**：\n    在终端进入项目目录，启动 Jupyter：\n    ```bash\n    jupyter-notebook\n    ```\n    在浏览器中选择对应的 `.ipynb` 文件打开。\n\n2.  **练习概览**：\n    打开 Notebook 后，你将看到 4 个难度递增的练习。数据集定义在 `datasets.py` 中，你需要根据练习要求调整神经网络的超参数或架构。\n\n    *   **练习 1 (Exercise 1)**：确定性信号预测。\n        *   **目标**：预测两条相关联的时间序列信号。\n        *   **操作**：调整超参数，使模型能够根据左侧历史数据完美预测右侧黄色区域的未来数据。\n    \n    *   **练习 2 (Exercise 2)**：复合正弦波预测。\n        *   **目标**：预测由两个不同波长和偏移量的正弦波叠加而成的单一信号。\n        *   **推荐初始参数**：\n            ```python\n            n_samples = 125000\n            epochs = 1\n            batch_size = 50\n            hidden_dim = 35\n            ```\n        *   **进阶**：尝试增加堆叠的循环细胞数量（如 3 层）和隐藏单元宽度（如 500）以获得更佳效果。\n\n    *   **练习 3 (Exercise 3)**：去噪自回归预测。\n        *   **目标**：输入为含噪信号，输出为平滑的未来值。模型需具备去噪能力。\n        *   **特点**：这是一个去噪自编码器（Denoising Autoencoder）的应用场景。\n\n    *   **练习 4 (Exercise 4)**：比特币价格预测（开放型挑战）。\n        *   **目标**：基于 BTC\u002FUSD 和 BTC\u002FEUR 的历史日线数据预测未来价格。\n        *   **扩展建议**：为了获得更准确的预测，建议在 `datasets.py` 中引入更多特征维度，例如：\n            *   其他金融指数（S&P 500, Dow Jones）。\n            *   时间特征（小时、星期、月份、月相周期等正弦\u002F余弦编码）。\n            *   外部数据（气象数据、社交媒体情感分析得分）。\n        *   **输入输出示例**：\n            *   Input: `(BTC\u002FUSD, BTC\u002FEUR, Dow_Jones, hour_of_day, ...)`\n            *   Output: `(BTC\u002FUSD, BTC\u002FEUR)`\n\n开始你的第一个练习，通过修改代码中的训练参数和網絡结构，观察预测曲线（黄色部分）如何逐渐逼近真实值。","某量化交易团队正在开发一套加密货币多币种联动预测系统，需要基于历史数据同时预判比特币（BTC）对美元和欧元的未来汇率走势。\n\n### 没有 seq2seq-signal-prediction 时\n- **多变量关联建模困难**：传统单输出模型难以捕捉 USD\u002FBTC 与 EUR\u002FBTC 两条时间序列之间复杂的动态耦合关系，往往只能单独预测，忽略了币种间的联动效应。\n- **长序列依赖丢失**：使用简单的回归方法无法有效记忆长期的历史波动模式，导致在面对具有周期性或长滞后特征的市场信号时，预测结果严重失真。\n- **架构调整成本高昂**：每当需要处理不同长度或复杂度的信号组合时，开发人员必须从头重写大量底层 TensorFlow 代码来调整网络结构，试错周期极长。\n- **缺乏确定性基准验证**：在没有类似 Exercise 1 这样的确定性信号练习作为基准的情况下，团队难以判断模型是未收敛还是架构本身存在缺陷，调试方向模糊。\n\n### 使用 seq2seq-signal-prediction 后\n- **原生支持多维序列输出**：利用其 Encoder-Decoder 架构，轻松实现输入过去多币种数据、同时输出未来多条关联曲线的功能，精准捕捉汇率间的协同变化。\n- **强大的时序特征提取**：基于 RNN 的序列到序列机制有效保留了长短期记忆，即使面对叠加了不同波长和偏移量的复杂正弦波信号（如 Exercise 2），也能还原出平滑准确的趋势。\n- **模块化实验流程**：直接复用项目中预设的四个渐进式练习框架，只需修改超参数或少量架构代码即可适配新场景，将模型迭代时间从数天缩短至数小时。\n- **可视化调试直观**：借助项目自带的图表生成逻辑，清晰对比“已知历史”与“黄色预测区”，快速定位模型在特定波形下的表现瓶颈，大幅降低调优门槛。\n\nseq2seq-signal-prediction 通过成熟的编码器 - 解码器范式，将复杂的多变量时间序列预测转化为可快速迭代的标准工程任务，显著提升了量化策略的研发效率。","https:\u002F\u002Foss.gittoolsai.com\u002Fimages\u002Fguillaume-chevalier_seq2seq-signal-prediction_2df3686c.png","guillaume-chevalier","Guillaume Chevalier","https:\u002F\u002Foss.gittoolsai.com\u002Favatars\u002Fguillaume-chevalier_15923461.png","e^(πi) + 1 = 0",null,"Canada","guillaume-chevalier@outlook.com","https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier",[82,86],{"name":83,"color":84,"percentage":85},"Jupyter Notebook","#DA5B0B",96,{"name":87,"color":88,"percentage":89},"Python","#3572A5",4,1084,289,"2026-04-02T08:35:11","Apache-2.0","未说明","可选（推荐用于加速），需安装 tensorflow-gpu==2.1，具体显卡型号和显存大小未说明",{"notes":97,"python":98,"dependencies":99},"该项目主要作为 Jupyter Notebook (.ipynb) 或 Google Colab 环境下的练习教程。若在本地运行，需启动 Jupyter Notebook；若使用 Google Colab 并利用 GPU 加速，需在运行时设置中选择'GPU'。代码基于较旧的 TensorFlow 2.1 版本构建。","3.6+",[100,101,102],"tensorflow-gpu==2.1","neuraxle==0.3.1","neuraxle_tensorflow==0.1.0",[14],[105,106,107,108],"seq2seq","tensorflow","tensorflow-tutorials","python","2026-03-27T02:49:30.150509","2026-04-06T19:02:15.728732",[112,117,122,127,132,137,142,147],{"id":113,"question_zh":114,"answer_zh":115,"source_url":116},19664,"如何实现输入和输出序列具有不同的维度（例如用多个正弦波预测单个正弦波）？","可以通过使用 TensorFlow 的 Dynamic RNN 或在 Eager 模式下实现。该功能已在合并 PR #17 后得到修复和支持。","https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002Fseq2seq-signal-prediction\u002Fissues\u002F4",{"id":118,"question_zh":119,"answer_zh":120,"source_url":121},19665,"是否可以为时间序列数据实现可变长度的输入序列（类似机器翻译中的做法）？","可以。需要注意 input_seq_length 代表看到输入的次数，input_dim 代表每次看到的数值个数。如果将 input_seq_length 设为 1，问题将退化为前馈神经网络可解决的问题，不再需要循环神经网络（RNN），因为此时没有时间步的概念。","https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002Fseq2seq-signal-prediction\u002Fissues\u002F13",{"id":123,"question_zh":124,"answer_zh":125,"source_url":126},19666,"能否扩展程序以支持动态的输出大小（例如用 1000 个点预测 10 个点）？","可以使用 dynamic_rnn() 来实现动态输出大小。对于长序列预测（如 1000 步），建议在编码器部分引入 Phased LSTM (PLSTM) 架构以提高性能。具体实现可参考相关贡献指南或教程。","https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002Fseq2seq-signal-prediction\u002Fissues\u002F1",{"id":128,"question_zh":129,"answer_zh":130,"source_url":131},19667,"为什么在训练时解码器输入（dec_inp）使用零向量拼接编码器输入，而不是直接使用真实值或之前的预测值？","如果在训练时将解码器的输出反馈作为下一步的输入（Teacher Forcing 的变体），可能会导致奇怪的反馈循环和预测发散。除非在训练时也严格模拟这种反馈循环，否则通常建议在训练时使用真实值（Ground Truth），而在推理时使用上一步的预测值。","https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002Fseq2seq-signal-prediction\u002Fissues\u002F6",{"id":133,"question_zh":134,"answer_zh":135,"source_url":136},19668,"在 Jupyter Notebook 中多次运行训练单元格时出现\"Variable ... already exists\"错误，如何解决？","这是 TensorFlow 1.x 中常见的作用域复用问题。该项目已更新重构为 TensorFlow 2 版本，解决了变量重复定义的问题。建议升级代码库并使用 TF v2 运行。","https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002Fseq2seq-signal-prediction\u002Fissues\u002F10",{"id":138,"question_zh":139,"answer_zh":140,"source_url":141},19669,"遇到\"'Tensorflow2ModelStep' object has no attribute 'name'\"错误该怎么办？","该问题是由于版本不匹配导致的。维护者已在几天前发布了新版本修复了此问题，请更新项目代码到最新版本即可解决。","https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002Fseq2seq-signal-prediction\u002Fissues\u002F21",{"id":143,"question_zh":144,"answer_zh":145,"source_url":146},19670,"代码中的 reshaped_inputs 变量有什么作用？似乎未被后续使用。","该变量在旧版本代码中存在，但在代码重构为 TensorFlow 2 版本后已不再相关或被移除。请使用最新的 TF v2 代码分支。","https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002Fseq2seq-signal-prediction\u002Fissues\u002F5",{"id":148,"question_zh":149,"answer_zh":150,"source_url":151},19671,"如果我有 2000 个数据点的单特征输入，可以将 input_seq_len 设为 1 而 input_dim 设为 2000 吗？这样做有什么区别？","虽然技术上可以这样设置且代码能运行，但这会改变模型的性质。input_seq_len=1 意味着没有时间步，模型将无法利用时间序列的前后依赖关系，退化为普通的前馈神经网络。为了发挥 seq2seq 模型处理时间序列的优势，应保持 input_seq_len 为时间步长（如 2000），input_dim 为特征数（如 1）。","https:\u002F\u002Fgithub.com\u002Fguillaume-chevalier\u002Fseq2seq-signal-prediction\u002Fissues\u002F12",[]]