HCI Lab2：Image Retrieval

Design Section

1. 项目文件说明

项目源码构成：其中只对发下来的rest-server.py与main.html进行了较大修改

1. server/image_vectorizer.py
Used to extract features from dataset and save it on the distination.

2. server/rest-server.py
Basic level of validation is also being done in this file. And it also sends data generated by the back-end to the front-end.

3. server/search.py
This file implements the image search/retrieva.

4. server/templates/main.html
This is the front-end interface I designed.

2. 关键代码分析

2.1. rest-server.py

• Flask 和其模块的导入
该代码使用 Flask 框架来创建 Web 应用程序。其中，需要从 Flask 模块中导入 Flask 类，以创建 Flask 应用程序对象。此外，还需要从 Flask 模块中导入其他模块和函数，如 jsonify、request、render_template 等用于实现不同功能的函数和类。
在该代码中，使用了 Flask 应用程序对象的 app.route 装饰器来定义了两个路由，分别是 '/' 和 '/imgUpload'。这两个路由都是 Flask 应用程序中的 URL 映射，用于将客户端发起的请求映射到对应的视图函数。
具体来说，'/' 路由对应的视图函数是名为 main 的函数，该函数使用了 Flask 应用程序对象的 render_template 方法来渲染名为 main.html 的模板文件。这个路由用于显示应用程序的主页。
'/imgUpload' 路由对应的视图函数是名为 upload_img 的函数，该函数用于处理客户端上传的图像并执行搜索。这个路由在前端代码中使用 Ajax 技术进行异步请求，从而实现无刷新上传和搜索的效果。
对应代码部分

• 从前端获取图片
使用Flask提供的request对象来获取前端上传的文件，这里用到了request.method，request.files，request.url，接着将上传的文件下载到了本地的文件夹中，使用完后还会将其删除

• 将后端产生的参数以JSON格式传给前端
使用 Flask 的 jsonify 函数将一个 Python 字典对象 para 转换为 JSON 对象，并作为 HTTP 响应返回给客户端。
para是一个字典，imagei标签用于指示对应的产生的图片检索结果的路径，featurei用于查找图像对应的标签，all_features用于输出9张图片中所有的标签

• 将图片标签提取出来的补充函数
通过使用find_features函数将根据输入的img_path找到其对应的特征(即出现在哪些.txt文件中)，最终以list的形式输出。 特征的集合如下：

2.2. main.html

• 导入外部库
导入了 Bootstrap、jQuery等库，以实现一些样式和功能
Bootstrap：用于实现页面布局和组件的样式，使用 Bootstrap 的类名（如 "navbar", "container", "table" 等）可以轻松地创建响应式布局和美观的组件。例如，在代码中使用了 "container"、"table" 和 "btn" 等 Bootstrap 类名
jQuery：jQuery 是一个快速、简洁的 JavaScript 库，用于简化 HTML 文档遍历、事件处理、动画和 Ajax 交互。在代码中，jQuery 主要用于操作 DOM 元素，例如显示或隐藏元素（$('#row0').show();、$('#load').hide();）、为表单提交事件添加处理程序（$("form").submit(function (evt) { ... }）以及发送Ajax请求【异步JS和XML，可以在不刷新的情况下更新局部页面的效果】（$.ajax({ ... });）。
详细代码

• 上传的文件接收（显示上传文件）

• 点击Search按钮后的处理（实现前后端互联）
其解决过程在fun()函数中得以实现：处理表单提交，并使用 AJAX 发送请求。注意，此处将上传的图片放到/imgUpload上与后端upload_img函数定义的/imgUpload路由一致，则可以直接通过后端的request.files['file']进行接收上传的文件。

• 对于9张图片的筛选
• 点击心形进行收藏
若点击了♥，则♥变色，同时将“selected”标签加入该♥的classList中（“selected”并不用实现），然后通过看是否有“selected”来得知是否被喜欢。
点击滑块后，会调用如下匿名函数，对图片的不透明度(opacity)与尺寸(scale)进行修改，该函数在html文档加载完成后执行，实时监控toggle-switch的状态，并不需要显式调用。【ps:改变尺寸会使得整个网站显得更加丝滑】
• 根据标签选择图片
标签会在点击完search的showFileName函数中初始化值；每一张图片的class中也会有其对应标签的名字。 每一个标签按钮都绑定了一个toggleButton函数，一开始标签处于选择状态，一旦取消该标签之后，会改变标签的颜色，同时使得含有该标签的图片的不透明度降低，以达到选择出标签的结果

3. 过程中遇到的问题及解决

• 格式发生变化
在解决通过滑块让图片消失的问题时，我直接使用$(”#img0”).hide使得图片确实消失了，但其下面的心形上移了，十分丑陋。
解决方案：改变图片的不透明度，使其变得透明（也可以直接opacity=0，直接变白）

• 牛头不对马嘴型
在最后的整合过程中，我重新运行了一遍所有代码（image_vectorizer.py应该没跑完），就出现了如下情况。



解决方案：再跑一次image_vectorizer.py就行啦！

Report Part

1. Requirements of an Image Search Task

• Upload an image as a query
This feature allows users to provide an image as input for the search engine, which then retrieves visually similar or relevant images based on the query. The system can accept various image formats and should handle potential errors, such as unsupported file types or large file sizes.

• Preview the query image
Once an image is uploaded, it should be displayed as a preview within the search interface. This allows users to verify that the correct image has been selected and to make adjustments if necessary. The preview should be a scaled-down version of the original image to save bandwidth and reduce loading time.

• Initiate the search using the query image
Users should be able to start the search process with a simple click of a button. The search engine then analyzes the query image and retrieves matching results based on its visual features, metadata, or other criteria. The search should be efficient and optimized to return relevant results in a timely manner.

• Review the search results with an overview and options to refine the search
The system should display a brief overview of the search results, including the total number of results and any relevant information. Users should be able to review the results in a user-friendly format, such as a grid or list view with thumbnail previews, image titles, and other metadata. Additionally, users should be able to refine their search by modifying parameters like categories, tags, or other filters, and sort the results by relevance, date, or other criteria.

• Interact with the results, such as adding images to a favorites list
Users should be able to perform various actions on the search results, including adding images to a favorites list or a custom collection, downloading images for personal use, and sharing images through email, social media, or other platforms. The system should also provide options for users to report inappropriate content or request more information about a particular image.

2. Page Overview

3. Design and Implementation of the Five-Stage Framework

3.1. Formulation Stage

• Include an input box for users to upload images: This input box should support uploading different types of images to provide a smooth user experience. Supported formats here include 'png', 'jpg' and 'jpeg'.

• Show a preview of the uploaded image in the search window: the preview should be a reduced version of the original image to save bandwidth and reduce loading time. Also, the name of the uploaded image can be displayed next to the input box to check if it has been uploaded correctly. If it is not correct you can click on the Choose a File button to re-upload it.

3.2. Initiation Stage

• Add a search button for users to initiate the search using the uploaded image: The Search button should be prominently displayed and easily accessible, enabling users to start the search process with a single click.

3.3. Review Stage

• Display an overview of the search results, including the total number of results found: This overview should be presented in a concise and easily understandable format, allowing users to quickly gauge the effectiveness of their search query.

• Display results in a grid format with thumbnail preview: Each result should be displayed as a thumbnail image. This format allows the user to quickly navigate through the search results and identify images of interest.

3.4. Refinement Stage

• Allow users to modify search parameters to refine their search results by selecting certain categories or tags: Users should have the option to narrow their search results by applying filters, such as specific categories or tags. These filters are easily accessible and clearly labelled.

3.5. Use Stage

• Enable users to interact with search results: Users can mark their favourite images by highlighting the hearts below them and view only their favourite images by clicking on the Just show what I like slider, which hides the unlit hearts. 【Additional】At the same time, the whole process will change the size of the image, making the interaction more fluid.

• 【Additional】Image display size can be modified: to make it easier for users to view the images and select their favourite ones, the images can be scaled by clicking on them.