国内python职位数据分析_flask+spider
- 开发语言：Python
- 框架：flask
- Python版本：python3.8
- 数据库：mysql 5.7
- 数据库工具：Navicat12
- 开发软件：PyCharm
系统展示
系统首页
招聘信息页面
管理员登录
管理员功能界面
用户管理
招聘信息管理
看板展示
论坛交流
系统管理
摘要
系统阐述的是使用国内python职位数据分析系统的设计与实现，对于Python、B/S结构、MySql进行了较为深入的学习与应用。主要针对系统的设计，描述，实现和分析与测试方面来表明开发的过程。开发中使用了 Flask框架和MySql数据库技术搭建系统的整体架构。利用这些技术结合实际需求开发了具有个人中心、用户管理、招聘信息管理、论坛交流、系统管理等功能的系统，最后对系统进行相应的测试，测试系统有无存在问题以及测试用户权限来优化系统，最后系统达到预期目标。
研究背景
中国在两千年左右就已经对计算机这一块发展到相当的规模了，现在老百姓们越来越离不开计算机网络、互联网所带来的好处了，现如今各种格式的网站系统遍地开花，现在不同于以往的老旧的管理方式了，只有跟上时代的发展才能不会被淘汰掉，所以对于国内python职位数据分析来说也是同样的，将传统的线下管理带到线上，能够大大提升管理效率，好处也有很多，首先整体的美化、提升了在新时代的背景之下新的样貌，更加朝气蓬勃。一个好的系统能够把传统的国内python职位数据分析信息管理方式，带上一全新的方式，效率、可靠、安全稳定集合一身的系统，带来的体验是传统管理方式所没有办法实现的，信息的交流将达到顶峰，用户的沟通将会更加的便捷，有问题实时反馈提交，及时接收重要通知，将国内python职位数据分析管理正规化、合理化、高效化。
关键技术
Python是解释型的脚本语言，在运行过程中，把程序转换为字节码和机器语言，说明性语言的程序在运行之前不必进行编译，而是一个专用的解释器，当被执行时，它都会被翻译，与之对应的还有编译性语言。
同时，这也是一种用于电脑编程的跨平台语言，这是一门将编译、交互和面向对象相结合的脚本语言（script language）。
Flask是一个使用Python编写的轻量级Web应用框架。它被称为一个“微框架”（microframework），因为它只提供Web应用所需的最核心的功能，如路由、会话管理和模板引擎等，而不像一些更全面的框架那样包含数据库层、表单处理等功能。然而，Flask的扩展生态系统非常丰富，开发者可以通过添加扩展来为Flask应用添加这些额外的功能。
Vue是一款流行的开源JavaScript框架，用于构建用户界面和单页面应用程序。Vue的核心库只关注视图层，易于上手并且可以与其他库或现有项目轻松整合。
MYSQL数据库运行速度快，安全性能也很高，而且对使用的平台没有任何的限制，所以被广泛应运到系统的开发中。MySQL是一个开源和多线程的关系管理数据库系统，MySQL是开放源代码的数据库，具有跨平台性。
B/S（浏览器/服务器）结构是目前主流的网络化的结构模式，它能够把系统核心功能集中在服务器上面，可以帮助系统开发人员简化操作，便于维护和使用。
系统分析
对系统的可行性分析以及对所有功能需求进行详细的分析，来查看该系统是否具有开发的可能。
系统设计
功能模块设计和数据库设计这两部分内容都有专门的表格和图片表示。
系统实现
当人们打开系统的网址后，首先看到的就是首页界面。在这里，人们能够看到系统的导航条，通过导航条导航进入各功能展示页面进行操作。管理员进入主页面，主要功能包括对个人中心、用户管理、招聘信息管理、论坛交流、系统管理等进行操作。
代码实现
# 注册接口 @main_bp.route("/pythonka8v0/zhaopinxinxi/register", methods=['POST']) def pythonka8v0_zhaopinxinxi_register(): if request.method == 'POST': msg = {'code': normal_code, 'message': 'success', 'data': [{}]} req_dict = session.get("req_dict") error = zhaopinxinxi.createbyreq(zhaopinxinxi, zhaopinxinxi, req_dict) if error!=None: msg['code'] = crud_error_code msg['msg'] = "注册用户已存在" return jsonify(msg) # 登录接口 @main_bp.route("/pythonka8v0/zhaopinxinxi/login", methods=['GET','POST']) def pythonka8v0_zhaopinxinxi_login(): if request.method == 'GET' or request.method == 'POST': msg = {"code": normal_code, "msg": "success", "data": {}} req_dict = session.get("req_dict") req_model = session.get("req_dict") try: del req_model['role'] except: pass datas = zhaopinxinxi.getbyparams(zhaopinxinxi, zhaopinxinxi, req_model) if not datas: msg['code'] = password_error_code msg['msg']='密码错误或用户不存在' return jsonify(msg) req_dict['id'] = datas[0].get('id') return Auth.authenticate(Auth, zhaopinxinxi, req_dict) # 登出接口 @main_bp.route("/pythonka8v0/zhaopinxinxi/logout", methods=['POST']) def pythonka8v0_zhaopinxinxi_logout(): if request.method == 'POST': msg = { "msg": "退出成功", "code": 0 } req_dict = session.get("req_dict") return jsonify(msg) # 重置密码接口 @main_bp.route("/pythonka8v0/zhaopinxinxi/resetPass", methods=['POST']) def pythonka8v0_zhaopinxinxi_resetpass(): ''' ''' if request.method == 'POST': msg = {"code": normal_code, "msg": "success"} req_dict = session.get("req_dict") if req_dict.get('mima') != None: req_dict['mima'] = '123456' error = zhaopinxinxi.updatebyparams(zhaopinxinxi, zhaopinxinxi, req_dict) if error != None: msg['code'] = crud_error_code msg['msg'] = error else: msg['msg'] = '密码已重置为：123456' return jsonify(msg) # 获取会话信息接口 @main_bp.route("/pythonka8v0/zhaopinxinxi/session", methods=['GET']) def pythonka8v0_zhaopinxinxi_session(): ''' ''' if request.method == 'GET': msg = {"code": normal_code, "data": {}} req_dict={"id":session.get('params').get("id")} msg['data'] = zhaopinxinxi.getbyparams(zhaopinxinxi, zhaopinxinxi, req_dict)[0] return jsonify(msg) # 分类接口（后端） @main_bp.route("/pythonka8v0/zhaopinxinxi/page", methods=['GET']) def pythonka8v0_zhaopinxinxi_page(): ''' ''' if request.method == 'GET': msg = {"code": normal_code, "msg": "success", "data":{"currPage":1,"totalPage":1,"total":1,"pageSize":10,"list":[]}} req_dict = session.get("req_dict") userinfo = session.get("params") try: __hasMessage__=zhaopinxinxi.__hasMessage__ except: __hasMessage__=None if __hasMessage__ and __hasMessage__!="否": tablename=session.get("tablename") if tablename!="users" and session.get("params")!=None and zhaopinxinxi!='chat': req_dict["userid"]=session.get("params").get("id") tablename=session.get("tablename") if tablename=="users" : try: pass except: pass else: mapping_str_to_object = {} for model in Base_model._decl_class_registry.values(): if hasattr(model, '__tablename__'): mapping_str_to_object[model.__tablename__] = model try: __isAdmin__=mapping_str_to_object[tablename].__isAdmin__ except: __isAdmin__=None if __isAdmin__!="是" and session.get("params")!=None: req_dict["userid"]=session.get("params").get("id") else: try: del req_dict["userid"] except: pass clause_args = [] or_clauses = or_(*clause_args) msg['data']['list'], msg['data']['currPage'], msg['data']['totalPage'], msg['data']['total'], \ msg['data']['pageSize'] = zhaopinxinxi.page(zhaopinxinxi, zhaopinxinxi, req_dict, or_clauses) return jsonify(msg) # 排序接口 @main_bp.route("/pythonka8v0/zhaopinxinxi/autoSort", methods=['GET']) def pythonka8v0_zhaopinxinxi_autosort(): ''' ''' if request.method == 'GET': msg = {"code": normal_code, "msg": "success", "data":{"currPage":1,"totalPage":1,"total":1,"pageSize":10,"list":[]}} req_dict = session.get("req_dict") req_dict['sort']='clicktime' req_dict['order']='desc' try: __browseClick__= zhaopinxinxi.__browseClick__ except: __browseClick__=None if __browseClick__ =='是': req_dict['sort']='clicknum' elif __browseClick__ =='时长': req_dict['sort']='browseduration' else: req_dict['sort']='clicktime' msg['data']['list'], msg['data']['currPage'], msg['data']['totalPage'], msg['data']['total'], \ msg['data']['pageSize'] = zhaopinxinxi.page(zhaopinxinxi, zhaopinxinxi, req_dict) return jsonify(msg) # 分页接口（前端） @main_bp.route("/pythonka8v0/zhaopinxinxi/list", methods=['GET']) def pythonka8v0_zhaopinxinxi_list(): ''' ''' if request.method == 'GET': msg = {"code": normal_code, "msg": "success", "data":{"currPage":1,"totalPage":1,"total":1,"pageSize":10,"list":[]}} req_dict = session.get("req_dict") if req_dict.__contains__('vipread'): del req_dict['vipread'] userinfo = session.get("params") try: __foreEndList__=zhaopinxinxi.__foreEndList__ except: __foreEndList__=None if __foreEndList__ and __foreEndList__!="否": tablename=session.get("tablename") if tablename!="users" and session.get("params")!=None: req_dict['userid']=session.get("params").get("id") try: __foreEndListAuth__=zhaopinxinxi.__foreEndListAuth__ except: __foreEndListAuth__=None if __foreEndListAuth__ and __foreEndListAuth__!="否": tablename=session.get("tablename") if tablename!="users" and session.get("params")!=None: req_dict['userid']=session.get("params").get("id") tablename=session.get("tablename") if tablename=="users" : try: del req_dict["userid"] except: pass else: mapping_str_to_object = {} for model in Base_model._decl_class_registry.values(): if hasattr(model, '__tablename__'): mapping_str_to_object[model.__tablename__] = model try: __isAdmin__=mapping_str_to_object[tablename].__isAdmin__ except: __isAdmin__=None if __isAdmin__!="是" and session.get("params")!=None: req_dict["userid"]=session.get("params").get("id") else: try: del req_dict["userid"] except: pass if 'luntan' in 'zhaopinxinxi': if 'userid' in req_dict.keys(): del req_dict["userid"] msg['data']['list'], msg['data']['currPage'], msg['data']['totalPage'], msg['data']['total'], \ msg['data']['pageSize'] = zhaopinxinxi.page(zhaopinxinxi, zhaopinxinxi, req_dict) return jsonify(msg) # 保存接口（后端） @main_bp.route("/pythonka8v0/zhaopinxinxi/save", methods=['POST']) def pythonka8v0_zhaopinxinxi_save(): ''' ''' if request.method == 'POST': msg = {"code": normal_code, "msg": "success", "data": {}} req_dict = session.get("req_dict") for key in req_dict: if req_dict[key] == '': req_dict[key] = None error= zhaopinxinxi.createbyreq(zhaopinxinxi, zhaopinxinxi, req_dict) if error!=None: msg['code'] = crud_error_code msg['msg'] = error return jsonify(msg) # 添加接口（前端） @main_bp.route("/pythonka8v0/zhaopinxinxi/add", methods=['POST']) def pythonka8v0_zhaopinxinxi_add(): ''' ''' if request.method == 'POST': msg = {"code": normal_code, "msg": "success", "data": {}} req_dict = session.get("req_dict") try: __foreEndListAuth__=zhaopinxinxi.__foreEndListAuth__ except: __foreEndListAuth__=None if __foreEndListAuth__ and __foreEndListAuth__!="否": tablename=session.get("tablename") if tablename!="users": req_dict['userid']=session.get("params").get("id") error= zhaopinxinxi.createbyreq(zhaopinxinxi, zhaopinxinxi, req_dict) if error!=None: msg['code'] = crud_error_code msg['msg'] = error return jsonify(msg) # 踩、赞接口 @main_bp.route("/pythonka8v0/zhaopinxinxi/thumbsup/<id_>", methods=['GET']) def pythonka8v0_zhaopinxinxi_thumbsup(id_): ''' ''' if request.method == 'GET': msg = {"code": normal_code, "msg": "success", "data": {}} req_dict = session.get("req_dict") id_=int(id_) type_=int(req_dict.get("type",0)) rets=zhaopinxinxi.getbyid(zhaopinxinxi, zhaopinxinxi,id_) update_dict={ "id":id_, } if type_==1:#赞 update_dict["thumbsupnum"]=int(rets[0].get('thumbsupnum'))+1 elif type_==2:#踩 update_dict["crazilynum"]=int(rets[0].get('crazilynum'))+1 error = zhaopinxinxi.updatebyparams(zhaopinxinxi, zhaopinxinxi, update_dict) if error!=None: msg['code'] = crud_error_code msg['msg'] = error return jsonify(msg) # 获取详情信息（后端） @main_bp.route("/pythonka8v0/zhaopinxinxi/info/<id_>", methods=['GET']) def pythonka8v0_zhaopinxinxi_info(id_): ''' ''' if request.method == 'GET': msg = {"code": normal_code, "msg": "success", "data": {}} data = zhaopinxinxi.getbyid(zhaopinxinxi, zhaopinxinxi, int(id_)) if len(data)>0: msg['data']=data[0] #浏览点击次数 try: __browseClick__= zhaopinxinxi.__browseClick__ except: __browseClick__=None if __browseClick__ and "clicknum" in zhaopinxinxi.__table__.columns: click_dict={"id":int(id_),"clicknum":str(int(data[0].get("clicknum") or 0)+1)} ret=zhaopinxinxi.updatebyparams(zhaopinxinxi,zhaopinxinxi,click_dict) if ret!=None: msg['code'] = crud_error_code msg['msg'] = ret return jsonify(msg) # 获取详情信息（前端） @main_bp.route("/pythonka8v0/zhaopinxinxi/detail/<id_>", methods=['GET']) def pythonka8v0_zhaopinxinxi_detail(id_): ''' ''' if request.method == 'GET': msg = {"code": normal_code, "msg": "success", "data": {}} data = zhaopinxinxi.getbyid(zhaopinxinxi, zhaopinxinxi, int(id_)) if len(data)>0: msg['data']=data[0] #浏览点击次数 try: __browseClick__= zhaopinxinxi.__browseClick__ except: __browseClick__=None if __browseClick__ and "clicknum" in zhaopinxinxi.__table__.columns: click_dict={"id":int(id_),"clicknum":str(int(data[0].get("clicknum") or 0)+1)} ret=zhaopinxinxi.updatebyparams(zhaopinxinxi,zhaopinxinxi,click_dict) if ret!=None: msg['code'] = crud_error_code msg['msg'] = ret return jsonify(msg) # 更新接口 @main_bp.route("/pythonka8v0/zhaopinxinxi/update", methods=['POST']) def pythonka8v0_zhaopinxinxi_update(): ''' ''' if request.method == 'POST': msg = {"code": normal_code, "msg": "success", "data": {}} req_dict = session.get("req_dict") if req_dict.get("mima") and "mima" not in zhaopinxinxi.__table__.columns : del req_dict["mima"] if req_dict.get("password") and "password" not in zhaopinxinxi.__table__.columns : del req_dict["password"] try: del req_dict["clicknum"] except: pass error = zhaopinxinxi.updatebyparams(zhaopinxinxi, zhaopinxinxi, req_dict) if error!=None: msg['code'] = crud_error_code msg['msg'] = error return jsonify(msg) # 删除接口 @main_bp.route("/pythonka8v0/zhaopinxinxi/delete", methods=['POST']) def pythonka8v0_zhaopinxinxi_delete(): ''' ''' if request.method == 'POST': msg = {"code": normal_code, "msg": "success", "data": {}} req_dict = session.get("req_dict") error=zhaopinxinxi.delete( zhaopinxinxi, req_dict ) if error!=None: msg['code'] = crud_error_code msg['msg'] = error return jsonify(msg) # 投票接口 @main_bp.route("/pythonka8v0/zhaopinxinxi/vote/<int:id_>", methods=['POST']) def pythonka8v0_zhaopinxinxi_vote(id_): ''' ''' if request.method == 'POST': msg = {"code": normal_code, "msg": "success"} data= zhaopinxinxi.getbyid(zhaopinxinxi, zhaopinxinxi, int(id_)) for i in data: votenum=i.get('votenum') if votenum!=None: params={"id":int(id_),"votenum":votenum+1} error=zhaopinxinxi.updatebyparams(zhaopinxinxi,zhaopinxinxi,params) if error!=None: msg['code'] = crud_error_code msg['msg'] = error return jsonify(msg) # 分组统计接口 @main_bp.route("/pythonka8v0/zhaopinxinxi/group/<columnName>", methods=['GET']) def pythonka8v0_zhaopinxinxi_group(columnName): ''' 分组统计接口 ''' if request.method == 'GET': msg = {"code": normal_code, "msg": "success", "data": {}} req_dict = session.get("req_dict") userinfo = session.get("params") msg['data'] = zhaopinxinxi.groupbycolumnname(zhaopinxinxi,zhaopinxinxi,columnName,req_dict) msg['data'] = msg['data'][:10] return jsonify(msg) # 按值统计接口 @main_bp.route("/pythonka8v0/zhaopinxinxi/value/<xColumnName>/<yColumnName>", methods=['GET']) def pythonka8v0_zhaopinxinxi_value(xColumnName, yColumnName): ''' 按值统计接口, { "code": 0, "data": [ { "total": 10.0, "shangpinleibie": "aa" }, { "total": 20.0, "shangpinleibie": "bb" }, { "total": 15.0, "shangpinleibie": "cc" } ] } ''' if request.method == 'GET': msg = {"code": normal_code, "msg": "success", "data": {}} req_dict = session.get("req_dict") userinfo = session.get("params") msg['data'] = zhaopinxinxi.getvaluebyxycolumnname(zhaopinxinxi,zhaopinxinxi,xColumnName,yColumnName,req_dict) msg['data'] = msg['data'][:10] return jsonify(msg) # 按日期统计接口 @main_bp.route("/pythonka8v0/zhaopinxinxi/value/<xColumnName>/<yColumnName>/<timeStatType>", methods=['GET']) def pythonka8v0_zhaopinxinxi_value_riqi(xColumnName, yColumnName, timeStatType): ''' 按日期统计接口 ''' if request.method == 'GET': msg = {"code": normal_code, "msg": "success", "data": {}} where = ' where 1 = 1 ' if timeStatType == '日': sql = "SELECT DATE_FORMAT({0}, '%Y-%m-%d') {0}, sum({1}) total FROM zhaopinxinxi {2} GROUP BY DATE_FORMAT({0}, '%Y-%m-%d')".format(xColumnName, yColumnName, where, '%Y-%m-%d') if timeStatType == '月': sql = "SELECT DATE_FORMAT({0}, '%Y-%m') {0}, sum({1}) total FROM zhaopinxinxi {2} GROUP BY DATE_FORMAT({0}, '%Y-%m')".format(xColumnName, yColumnName, where, '%Y-%m') if timeStatType == '年': sql = "SELECT DATE_FORMAT({0}, '%Y') {0}, sum({1}) total FROM zhaopinxinxi {2} GROUP BY DATE_FORMAT({0}, '%Y')".format(xColumnName, yColumnName, where, '%Y') data = db.session.execute(sql) data = data.fetchall() results = [] for i in range(len(data)): result = { xColumnName: decimalEncoder(data[i][0]), 'total': decimalEncoder(data[i][1]) } results.append(result) msg['data'] = results return jsonify(msg) # 推荐算法接口 @main_bp.route("/pythonka8v0/zhaopinxinxi/autoSort2", methods=['GET']) def pythonka8v0_zhaopinxinxi_autoSort2(): if request.method == 'GET': leixing = set() req_dict = session.get("req_dict") userinfo = session.get("params") sql = "select inteltype from storeup where userid = "+userinfo.get("id")+" and tablename = 'zhaopinxinxi' order by addtime desc" try: data = db.session.execute(sql) rows = data.fetchall() for row in rows: for item in row: if item != None: leixing.add(item) except: leixing = set() L = [] sql ="select * from zhaopinxinxi where $intelRecomColumn in ('%s"%("','").join(leixing)+"') union all select * from zhaopinxinxi where $intelRecomColumn not in('%s"%("','").join(leixing)+"')" data = db.session.execute(sql) data_dict = [dict(zip(result.keys(), result)) for result in data.fetchall()] for online_dict in data_dict: for key in online_dict: if 'datetime.datetime' in str(type(online_dict[key])): online_dict[key] = online_dict[key].strftime( "%Y-%m-%d %H:%M:%S") else: pass L.append(online_dict) return jsonify({"code": 0, "msg": '', "data":{"currPage":1,"totalPage":1,"total":1,"pageSize":5,"list": L[0:int(req_dict['limit'])]}}) # 总数量 @main_bp.route("/pythonka8v0/zhaopinxinxi/count", methods=['GET']) def pythonka8v0_zhaopinxinxi_count(): ''' ''' if request.method == 'GET': msg = {"code": normal_code, "msg": "success", "data": 0} req_dict = session.get("req_dict") userinfo = session.get("params") msg['data'] = zhaopinxinxi.count(zhaopinxinxi, zhaopinxinxi, req_dict) return jsonify(msg) # 统计接口 @main_bp.route("/pythonka8v0/zhaopinxinxi/remind/<columnName>/<type>", methods=['GET']) # def pythonka8v0_zhaopinxinxi_remind(columnName,type): ''' ''' if request.method == 'GET': msg = {"code": normal_code, 'count': 0} # 组合查询参数 params = session.get("req_dict") if int(type)==1:#数字 if params.get('remindstart') == None and params.get('remindend') != None: remindstart = 0 remindend = int(params['remindend']) elif params.get('remindstart') != None and params.get('remindend') == None: remindstart = int(params['remindstart']) remindend = 999999 elif params.get('remindstart') == None and params.get('remindend') == None: remindstart = 0 remindend = 999999 elif int(type)==2:#日期 current_time=int(time.time()) if params.get('remindstart') == None and params.get('remindend') != None: starttime=current_time-60*60*24*365*2 params['remindstart'] = time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(starttime)) endtime=current_time+60*60*24*params.get('remindend') params['remindend'] = time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(endtime)) elif params.get('remindstart') != None and params.get('remindend') == None: starttime= current_time - 60 * 60 * 24 * params.get('remindstart') params['remindstart']=time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(starttime)) endtime=current_time+60*60*24*365*2 params['remindend'] = time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(endtime)) elif params.get('remindstart') == None and params.get('remindend') == None: starttime = current_time - 60 * 60 * 24 * 365 * 2 params['remindstart'] = time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(starttime)) endtime = current_time + 60 * 60 * 24 * 365 * 2 params['remindend'] = time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(endtime)) data = zhaopinxinxi.getbetweenparams( zhaopinxinxi, zhaopinxinxi, columnName, { "remindStart": remindstart, "remindEnd": remindend } ) msg['count'] = len(data) return jsonify(msg)系统测试
系统测试（System Testing）是为给客户提供测试产品或服务的质量信息而进行的检查。系统测试还可以提供客观和独立的系统评估，以使运营者能够了解和系统实施所面临的潜在问题。系统测试包括（软件的组件，系统组件）的执行，以评估系统属性。通常这些属性表明被测组件或系统满足系统预期开发需求，在各种预期的时间内，正确响应各种系统输入，同时可以满足分析设计时要求的程度。在预期的环境中运行，并达到用户期望的总体结果。经过一系列严格功能测试，以发现系统功能方面潜在的问题，保证系统的正常运行。
结论
系统的开发使我能够结合我以前的知识集成和改进它们。毕设就是检验我们对所学内容运用程度的时刻，对所学内容运用情况进行巩固与提高。本系统检验了自己的IT知识，同时也检验了自己运用相关知识去解决实际问题，使自己得到了一个锻炼与提升编程技能的机会。在设计上我遇到了很多困难，主要原因是我所拥有的知识不是很扎实，所以只靠几本书里的知识是远远不够的！自己经过实践取得了一些实践经验。知识不在于理解，必须学会运用，并且使之反映到实际生活中去，才是学知识的根本目的。面对困难，我学着怎样去处理，怎样去发现并分析有关的资料。
