Git Product home page Git Product logo

ali_jstorm's Introduction

ali_jstorm

使用阿里中间件RocketMQ、Tair、jstorm对双十一实时交易进行实时计算

具体内容如下: 2016阿里中间件性能挑战赛,初赛赛题:

第一赛季赛题

题目:模拟实时统计阿里双十一交易数据 赛题描述:我们将脱敏后的双11交易数据存储在RocketMQ中,选手利用JStorm从RocketMQ拉取数据,按照要求实时计算出结果,并将结果写入Tair。我们约定用整分时刻对应的时间戳来标识这一分钟,比如2015/11/11 08:11:00分钟对应的整分时间戳为1447200660,那么我们就用这个时间戳来表示这一分钟,即2015/11/11 08:11:00~2015/11/11 08:12::00(不包含这一时刻)。 选手需要计算的统计值如下: 1、分平台(淘宝、天猫)每分钟的交易金额,即计算每一分钟内的交易值。比如计算2015/11/11 11:11:00这一分钟的淘宝交易金额,就是统计从11:11:00分开始到11:12:00(不包含该时刻)这一分钟内的淘宝的交易总金额; 2、每整分时刻无线和PC端总交易金额比值;注意这里统计的是整分时刻对应的总交易比值,要将过去发生的所有交易都计算进去。比如计算2015/11/11 11:11:00整分时刻的交易比值,那么需要统计从交易开始时间到2015/11/11 11:12:00(不包含该时刻)这一区间内所有的交易值,来计算无线和PC端的比值; 最终存储到Tair 中的数据是key-value 的结构,key字符串格式, value是number类型。 key 统一以“固定前缀_teamcode_整分时间戳”方式命名的字符串,整分时间戳就是整分时刻对应的时间戳,例如2015/11/11 08:11:00分钟对应的时间戳为1447200660(注意是10位数),即该时刻的整分时间戳。其中teamcode是每支队伍唯一的标识。 (1)淘宝每分钟的交易金额的key是“platformTaobao_teamcode_整分时间戳”,值都是number类型;例如存入Tair的分平台淘宝实时交易金额格式如下:{"platformTaobao_1234auu_1447200660": 100} (2)天猫每分钟的交易金额的key是“platformTmall_ teamcode 整分时间戳”,值都是number类型;例如存入Tair的分平台天猫实时交易金额格式如下:{"platformTmall_1234auu_1447200660": 200} (3) 每整分时刻无线和PC端总交易金额比值的key为“ratio teamcode_整分时间戳”, 值是一个保留两位小数的number类型。例如存入Tair的无线和PC端实时交易金额比值格式如下:{"ratio_1234auu_1447200660": 2.12} 最终选手提交的代码,会跑在我们后台部署的集群上,我们后台部署的产品版本是:JStorm-2.1.1,RocketMQ-3.2.6,Tair-2.3.5。

ali_jstorm's People

Contributors

jiangchao123 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.