icesuns / icesuns.github.io Goto Github PK
View Code? Open in Web Editor NEWwelcome to my homepage
Home Page: https://zicesun.com
welcome to my homepage
Home Page: https://zicesun.com
<!DOCTYPE html> <html class="theme-next pisces use-motion" lang="zh-Hans"> <head><meta name="generator" content="Hexo 3.8.0"> <meta charset="UTF-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1"> <meta name="theme-color" content="#222"> <script src="/lib/pace/pace.min.js?v=1.0.2"></script> <link href="/lib/pace/pace-theme-minimal.min.css?v=1.0.2" rel="stylesheet"> <meta http-equiv="Cache-Control" content="no-transform"> <meta http-equiv="Cache-Control" content="no-siteapp"> <link href="/lib/fancybox/source/jquery.fancybox.css?v=2.1.5" rel="stylesheet" type="text/css"> <link href="/lib/font-awesome/css/font-awesome.min.css?v=4.6.2" rel="stylesheet" type="text/css"> <link href="/css/main.css?v=5.1.4" rel="stylesheet" type="text/css"> <link rel="apple-touch-icon" sizes="180x180" href="/images/apple-touch-icon-next.png?v=5.1.4"> <link rel="icon" type="image/png" sizes="32x32" href="/images/favicon-32x32-next.png?v=5.1.4"> <link rel="icon" type="image/png" sizes="16x16" href="/images/favicon-16x16-next.png?v=5.1.4"> <link rel="mask-icon" href="/images/logo.svg?v=5.1.4" color="#222"> <meta name="keywords" content="Hexo, NexT"> <meta name="description" content="welcome"> <meta property="og:type" content="website"> <meta property="og:title" content="ZIcesun"> <meta property="og:url" content="http://www.zicesun.com/README.html"> <meta property="og:site_name" content="ZIcesun"> <meta property="og:description" content="welcome"> <meta property="og:locale" content="zh-Hans"> <meta property="og:updated_time" content="2019-03-28T02:51:59.000Z"> <meta name="twitter:card" content="summary"> <meta name="twitter:title" content="ZIcesun"> <meta name="twitter:description" content="welcome"> <script type="text/javascript" id="hexo.configurations"> var NexT = window.NexT || {}; var CONFIG = { root: '/', scheme: 'Pisces', version: '5.1.4', sidebar: {"position":"left","display":"post","offset":12,"b2t":false,"scrollpercent":true,"onmobile":false}, fancybox: true, tabs: true, motion: {"enable":true,"async":false,"transition":{"post_block":"fadeIn","post_header":"slideDownIn","post_body":"slideDownIn","coll_header":"slideLeftIn","sidebar":"slideUpIn"}}, duoshuo: { userId: '0', author: '博主' }, algolia: { applicationID: '', apiKey: '', indexName: '', hits: {"per_page":10}, labels: {"input_placeholder":"Search for Posts","hits_empty":"We didn't find any results for the search: ${query}","hits_stats":"${hits} results found in ${time} ms"} } }; </script> <link rel="canonical" href="http://www.zicesun.com/README.html"> <!-- Global site tag (gtag.js) - Google Analytics --> <script async src="https://www.googletagmanager.com/gtag/js?id=UA-116658050-2"></script> <script> window.dataLayer = window.dataLayer || []; function gtag(){dataLayer.push(arguments);} gtag('js', new Date()); gtag('config', 'UA-116658050-2'); </script> <title> | ZIcesun</title> <script> (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) })(window,document,'script','https://www.google-analytics.com/analytics.js','ga'); ga('create', 'UA-116658050-1', 'auto'); ga('send', 'pageview'); </script> </head> <body itemscope itemtype="http://schema.org/WebPage" lang="zh-Hans"> <div class="container sidebar-position-left page-post-detail"> <div class="headband"></div> <header id="header" class="header" itemscope itemtype="http://schema.org/WPHeader"> <div class="header-inner"><div class="site-brand-wrapper"> <div class="site-meta "> <div class="custom-logo-site-title"> <a href="/" class="brand" rel="start"> <span class="logo-line-before"><i></i></span> <span class="site-title">ZIcesun</span> <span class="logo-line-after"><i></i></span> </a> </div> <h1 class="site-subtitle" itemprop="description">Live free or die</h1> </div> <div class="site-nav-toggle"> <button> <span class="btn-bar"></span> <span class="btn-bar"></span> <span class="btn-bar"></span> </button> </div> </div> <nav class="site-nav"> <ul id="menu" class="menu"> <li class="menu-item menu-item-home"> <a href="/" rel="section"> <i class="menu-item-icon fa fa-fw fa-home"></i> <br> 首页 </a> </li> <li class="menu-item menu-item-about"> <a href="/about/" rel="section"> <i class="menu-item-icon fa fa-fw fa-user"></i> <br> 关于 </a> </li> <li class="menu-item menu-item-tags"> <a href="/tags/" rel="section"> <i class="menu-item-icon fa fa-fw fa-tags"></i> <br> 标签 </a> </li> <li class="menu-item menu-item-categories"> <a href="/categories/" rel="section"> <i class="menu-item-icon fa fa-fw fa-th"></i> <br> 分类 </a> </li> <li class="menu-item menu-item-archives"> <a href="/archives/" rel="section"> <i class="menu-item-icon fa fa-fw fa-archive"></i> <br> 归档 </a> </li> <li class="menu-item menu-item-talking"> <a href="/talking" rel="section"> <i class="menu-item-icon fa fa-fw fa-plane"></i> <br> 说说 </a> </li> <li class="menu-item menu-item-essay"> <a href="/categories/essay/" rel="section"> <i class="menu-item-icon fa fa-fw fa-book"></i> <br> 随笔 </a> </li> <li class="menu-item menu-item-search"> <a href="javascript:;" class="popup-trigger"> <i class="menu-item-icon fa fa-search fa-fw"></i> <br> 搜索 </a> </li> </ul> <div class="site-search"> <div class="popup search-popup local-search-popup"> <div class="local-search-header clearfix"> <span class="search-icon"> <i class="fa fa-search"></i> </span> <span class="popup-btn-close"> <i class="fa fa-times-circle"></i> </span> <div class="local-search-input-wrapper"> <input autocomplete="off" placeholder="搜索..." spellcheck="false" type="text" id="local-search-input"> </div> </div> <div id="local-search-result"></div> </div> </div> </nav> </div> </header> <main id="main" class="main"> <div class="main-inner"> <div class="content-wrap"> <div id="content" class="content"> <div id="posts" class="posts-expand"> <div class="post-block page"> <header class="post-header"> <h2 class="post-title" itemprop="name headline"></h2> </header> <div class="post-body"> <p>welcome</p> </div> </div> </div> </div> <div id="gitalk-container"></div> </div> <div class="sidebar-toggle"> <div class="sidebar-toggle-line-wrap"> <span class="sidebar-toggle-line sidebar-toggle-line-first"></span> <span class="sidebar-toggle-line sidebar-toggle-line-middle"></span> <span class="sidebar-toggle-line sidebar-toggle-line-last"></span> </div> </div> <aside id="sidebar" class="sidebar"> <div class="sidebar-inner"> <section class="site-overview-wrap sidebar-panel sidebar-panel-active"> <div class="site-overview"> <div class="site-author motion-element" itemprop="author" itemscope itemtype="http://schema.org/Person"> <img class="site-author-image" itemprop="image" src="/images/avatar.jpg" alt="ZIcesun"> <p class="site-author-name" itemprop="name">ZIcesun</p> <p class="site-description motion-element" itemprop="description"></p> </div> <nav class="site-state motion-element"> <div class="site-state-item site-state-posts"> <a href="/archives/"> <span class="site-state-item-count">32</span> <span class="site-state-item-name">日志</span> </a> </div> <div class="site-state-item site-state-categories"> <a href="/categories/index.html"> <span class="site-state-item-count">14</span> <span class="site-state-item-name">分类</span> </a> </div> <div class="site-state-item site-state-tags"> <a href="/tags/index.html"> <span class="site-state-item-count">38</span> <span class="site-state-item-name">标签</span> </a> </div> </nav> <div class="links-of-author motion-element"> <span class="links-of-author-item"> <a href="https://github.com/icesuns" target="_blank" title="GitHub"> <i class="fa fa-fw fa-github"></i>GitHub</a> </span> <span class="links-of-author-item"> <a href="mailto:[email protected]" target="_blank" title="E-Mail"> <i class="fa fa-fw fa-envelope"></i>E-Mail</a> </span> <span class="links-of-author-item"> <a href="https://weibo.com/u/3902621880" target="_blank" title="微博"> <i class="fa fa-fw fa-weibo"></i>微博</a> </span> <span class="links-of-author-item"> <a href="http://www.zhihu.com/people/icesuns" target="_blank" title="知乎"> <i class="fa fa-fw fa-book"></i>知乎</a> </span> </div> <div class="links-of-blogroll motion-element links-of-blogroll-block"> <div class="links-of-blogroll-title"> <i class="fa fa-fw fa-link"></i> 友情链接 </div> <ul class="links-of-blogroll-list"> <li class="links-of-blogroll-item"> <a href="https://blog.csdn.net/Icesuns" title="icesuns" target="_blank">icesuns</a> </li> <li class="links-of-blogroll-item"> <a href="https://www.cnblogs.com/qingyunzong" title="扎心了,老铁" target="_blank">扎心了,老铁</a> </li> </ul> </div> </div> </section> </div> </aside> </div> </main> <footer id="footer" class="footer"> <div class="footer-inner"> <div class="copyright">© <span itemprop="copyrightYear">2021</span> <span class="with-love"> <i class="fa fa-user"></i> </span> <span class="author" itemprop="copyrightHolder">ZIcesun</span> <span class="post-meta-divider">|</span> <span class="post-meta-item-icon"> <i class="fa fa-area-chart"></i> </span> <span class="post-meta-item-text">Site words total count:</span> <span title="Site words total count">34.7k</span> </div> <div class="theme-info">主题 — <a class="theme-link" target="_blank" href="https://github.com/iissnan/hexo-theme-next">NexT.Pisces</a> v5.1.4</div> | 本页点击 <span id="busuanzi_value_page_pv"></span> 次 | 本站总点击 <span id="busuanzi_value_site_pv"></span> 次 | 您是第 <span id="busuanzi_value_site_uv"></span> 位访客 <script async src="https://dn-lbstatics.qbox.me/busuanzi/2.3/busuanzi.pure.mini.js"> </script> <div class="busuanzi-count"> <script async src="https://busuanzi.ibruce.info/busuanzi/2.3/busuanzi.pure.mini.js"></script> </div> </div> </footer> <div class="back-to-top"> <i class="fa fa-arrow-up"></i> <span id="scrollpercent"><span>0</span>%</span> </div> </div> <script type="text/javascript"> if (Object.prototype.toString.call(window.Promise) !== '[object Function]') { window.Promise = null; } </script> <script type="text/javascript" src="/lib/jquery/index.js?v=2.1.3"></script> <script type="text/javascript" src="/lib/fastclick/lib/fastclick.min.js?v=1.0.6"></script> <script type="text/javascript" src="/lib/jquery_lazyload/jquery.lazyload.js?v=1.9.7"></script> <script type="text/javascript" src="/lib/velocity/velocity.min.js?v=1.2.1"></script> <script type="text/javascript" src="/lib/velocity/velocity.ui.min.js?v=1.2.1"></script> <script type="text/javascript" src="/lib/fancybox/source/jquery.fancybox.pack.js?v=2.1.5"></script> <script type="text/javascript" src="/lib/canvas-nest/canvas-nest.min.js"></script> <script type="text/javascript" src="/js/src/utils.js?v=5.1.4"></script> <script type="text/javascript" src="/js/src/motion.js?v=5.1.4"></script> <script type="text/javascript" src="/js/src/affix.js?v=5.1.4"></script> <script type="text/javascript" src="/js/src/schemes/pisces.js?v=5.1.4"></script> <script type="text/javascript" src="/js/src/scrollspy.js?v=5.1.4"></script> <script type="text/javascript" src="/js/src/post-details.js?v=5.1.4"></script> <script type="text/javascript" src="/js/src/bootstrap.js?v=5.1.4"></script> <link rel="stylesheet" href="https://unpkg.com/gitalk/dist/gitalk.css"> <script src="https://unpkg.com/gitalk/dist/gitalk.min.js"></script> <script type="text/javascript"> var gitalk = new Gitalk({ clientID: '0a25b655a2577489330f', clientSecret: '63507e39d6d6c35ed2a33d6961ce748a0d64796d', repo: 'icesuns.github.io', owner: 'icesuns', admin: ['icesuns'], id: location.pathname, distractionFreeMode: 'true' }) gitalk.render('gitalk-container') </script> <script type="text/javascript"> // Popup Window; var isfetched = false; var isXml = true; // Search DB path; var search_path = "search.xml"; if (search_path.length === 0) { search_path = "search.xml"; } else if (/json$/i.test(search_path)) { isXml = false; } var path = "/" + search_path; // monitor main search box; var onPopupClose = function (e) { $('.popup').hide(); $('#local-search-input').val(''); $('.search-result-list').remove(); $('#no-result').remove(); $(".local-search-pop-overlay").remove(); $('body').css('overflow', ''); } function proceedsearch() { $("body") .append('<div class="search-popup-overlay local-search-pop-overlay"></div>') .css('overflow', 'hidden'); $('.search-popup-overlay').click(onPopupClose); $('.popup').toggle(); var $localSearchInput = $('#local-search-input'); $localSearchInput.attr("autocapitalize", "none"); $localSearchInput.attr("autocorrect", "off"); $localSearchInput.focus(); } // search function; var searchFunc = function(path, search_id, content_id) { 'use strict'; // start loading animation $("body") .append('<div class="search-popup-overlay local-search-pop-overlay">' + '<div id="search-loading-icon">' + '<i class="fa fa-spinner fa-pulse fa-5x fa-fw"></i>' + '</div>' + '</div>') .css('overflow', 'hidden'); $("#search-loading-icon").css('margin', '20% auto 0 auto').css('text-align', 'center'); $.ajax({ url: path, dataType: isXml ? "xml" : "json", async: true, success: function(res) { // get the contents from search data isfetched = true; $('.popup').detach().appendTo('.header-inner'); var datas = isXml ? $("entry", res).map(function() { return { title: $("title", this).text(), content: $("content",this).text(), url: $("url" , this).text() }; }).get() : res; var input = document.getElementById(search_id); var resultContent = document.getElementById(content_id); var inputEventFunction = function() { var searchText = input.value.trim().toLowerCase(); var keywords = searchText.split(/[\s\-]+/); if (keywords.length > 1) { keywords.push(searchText); } var resultItems = []; if (searchText.length > 0) { // perform local searching datas.forEach(function(data) { var isMatch = false; var hitCount = 0; var searchTextCount = 0; var title = data.title.trim(); var titleInLowerCase = title.toLowerCase(); var content = data.content.trim().replace(/<[^>]+>/g,""); var contentInLowerCase = content.toLowerCase(); var articleUrl = decodeURIComponent(data.url); var indexOfTitle = []; var indexOfContent = []; // only match articles with not empty titles if(title != '') { keywords.forEach(function(keyword) { function getIndexByWord(word, text, caseSensitive) { var wordLen = word.length; if (wordLen === 0) { return []; } var startPosition = 0, position = [], index = []; if (!caseSensitive) { text = text.toLowerCase(); word = word.toLowerCase(); } while ((position = text.indexOf(word, startPosition)) > -1) { index.push({position: position, word: word}); startPosition = position + wordLen; } return index; } indexOfTitle = indexOfTitle.concat(getIndexByWord(keyword, titleInLowerCase, false)); indexOfContent = indexOfContent.concat(getIndexByWord(keyword, contentInLowerCase, false)); }); if (indexOfTitle.length > 0 || indexOfContent.length > 0) { isMatch = true; hitCount = indexOfTitle.length + indexOfContent.length; } } // show search results if (isMatch) { // sort index by position of keyword [indexOfTitle, indexOfContent].forEach(function (index) { index.sort(function (itemLeft, itemRight) { if (itemRight.position !== itemLeft.position) { return itemRight.position - itemLeft.position; } else { return itemLeft.word.length - itemRight.word.length; } }); }); // merge hits into slices function mergeIntoSlice(text, start, end, index) { var item = index[index.length - 1]; var position = item.position; var word = item.word; var hits = []; var searchTextCountInSlice = 0; while (position + word.length <= end && index.length != 0) { if (word === searchText) { searchTextCountInSlice++; } hits.push({position: position, length: word.length}); var wordEnd = position + word.length; // move to next position of hit index.pop(); while (index.length != 0) { item = index[index.length - 1]; position = item.position; word = item.word; if (wordEnd > position) { index.pop(); } else { break; } } } searchTextCount += searchTextCountInSlice; return { hits: hits, start: start, end: end, searchTextCount: searchTextCountInSlice }; } var slicesOfTitle = []; if (indexOfTitle.length != 0) { slicesOfTitle.push(mergeIntoSlice(title, 0, title.length, indexOfTitle)); } var slicesOfContent = []; while (indexOfContent.length != 0) { var item = indexOfContent[indexOfContent.length - 1]; var position = item.position; var word = item.word; // cut out 100 characters var start = position - 20; var end = position + 80; if(start < 0){ start = 0; } if (end < position + word.length) { end = position + word.length; } if(end > content.length){ end = content.length; } slicesOfContent.push(mergeIntoSlice(content, start, end, indexOfContent)); } // sort slices in content by search text's count and hits' count slicesOfContent.sort(function (sliceLeft, sliceRight) { if (sliceLeft.searchTextCount !== sliceRight.searchTextCount) { return sliceRight.searchTextCount - sliceLeft.searchTextCount; } else if (sliceLeft.hits.length !== sliceRight.hits.length) { return sliceRight.hits.length - sliceLeft.hits.length; } else { return sliceLeft.start - sliceRight.start; } }); // select top N slices in content var upperBound = parseInt('1'); if (upperBound >= 0) { slicesOfContent = slicesOfContent.slice(0, upperBound); } // highlight title and content function highlightKeyword(text, slice) { var result = ''; var prevEnd = slice.start; slice.hits.forEach(function (hit) { result += text.substring(prevEnd, hit.position); var end = hit.position + hit.length; result += '<b class="search-keyword">' + text.substring(hit.position, end) + '</b>'; prevEnd = end; }); result += text.substring(prevEnd, slice.end); return result; } var resultItem = ''; if (slicesOfTitle.length != 0) { resultItem += "<li><a href='" + articleUrl + "' class='search-result-title'>" + highlightKeyword(title, slicesOfTitle[0]) + "</a>"; } else { resultItem += "<li><a href='" + articleUrl + "' class='search-result-title'>" + title + "</a>"; } slicesOfContent.forEach(function (slice) { resultItem += "<a href='" + articleUrl + "'>" + "<p class=\"search-result\">" + highlightKeyword(content, slice) + "...</p>" + "</a>"; }); resultItem += "</li>"; resultItems.push({ item: resultItem, searchTextCount: searchTextCount, hitCount: hitCount, id: resultItems.length }); } }) }; if (keywords.length === 1 && keywords[0] === "") { resultContent.innerHTML = '<div id="no-result"><i class="fa fa-search fa-5x" /></div>' } else if (resultItems.length === 0) { resultContent.innerHTML = '<div id="no-result"><i class="fa fa-frown-o fa-5x" /></div>' } else { resultItems.sort(function (resultLeft, resultRight) { if (resultLeft.searchTextCount !== resultRight.searchTextCount) { return resultRight.searchTextCount - resultLeft.searchTextCount; } else if (resultLeft.hitCount !== resultRight.hitCount) { return resultRight.hitCount - resultLeft.hitCount; } else { return resultRight.id - resultLeft.id; } }); var searchResultList = '<ul class=\"search-result-list\">'; resultItems.forEach(function (result) { searchResultList += result.item; }) searchResultList += "</ul>"; resultContent.innerHTML = searchResultList; } } if ('auto' === 'auto') { input.addEventListener('input', inputEventFunction); } else { $('.search-icon').click(inputEventFunction); input.addEventListener('keypress', function (event) { if (event.keyCode === 13) { inputEventFunction(); } }); } // remove loading animation $(".local-search-pop-overlay").remove(); $('body').css('overflow', ''); proceedsearch(); } }); } // handle and trigger popup window; $('.popup-trigger').click(function(e) { e.stopPropagation(); if (isfetched === false) { searchFunc(path, 'local-search-input', 'local-search-result'); } else { proceedsearch(); }; }); $('.popup-btn-close').click(onPopupClose); $('.popup').click(function(e){ e.stopPropagation(); }); $(document).on('keyup', function (event) { var shouldDismissSearchPopup = event.which === 27 && $('.search-popup').is(':visible'); if (shouldDismissSearchPopup) { onPopupClose(); } }); </script> <script> (function(){ var bp = document.createElement('script'); var curProtocol = window.location.protocol.split(':')[0]; if (curProtocol === 'https') { bp.src = 'https://zz.bdstatic.com/linksubmit/push.js'; } else { bp.src = 'http://push.zhanzhang.baidu.com/push.js'; } var s = document.getElementsByTagName("script")[0]; s.parentNode.insertBefore(bp, s); })(); </script> <script type="text/x-mathjax-config"> MathJax.Hub.Config({ tex2jax: { inlineMath: [ ['$','$'], ["\\(","\\)"] ], processEscapes: true, skipTags: ['script', 'noscript', 'style', 'textarea', 'pre', 'code'] } }); </script> <script type="text/x-mathjax-config"> MathJax.Hub.Queue(function() { var all = MathJax.Hub.getAllJax(), i; for (i=0; i < all.length; i += 1) { all[i].SourceElement().parentNode.className += ' has-jax'; } }); </script> <script type="text/javascript" src="//cdn.bootcss.com/mathjax/2.7.1/latest.js?config=TeX-AMS-MML_HTMLorMML"></script> </body> </html>
http://zicesun.com/2019/06/11/201906112027-spark-java-2/
本篇文章用spark实现了用户浏览购物网站的流处理。
http://zicesun.com/2018/04/03/201803272000-publish-first-blog/
搞了一下午和一晚上,博客终于弄好了,现在来测试一下。献上一首很喜欢的诗《致橡树》
http://zicesun.com/2019/03/27/1903271913-spark-learning-1/#more
之前看了一些Spark的相关内容,我觉得很有必要进行总结一下,不然都搭不起自己的知识框架。
https://zicesun.com/2019/03/22/Batch-Processing-vs-Stream-Processing/
最近在学习关于大数据处理方面的东西,因此想总结一下批处理和流处理之间的异同之处。
http://zicesun.com/2018/06/03/201803272000-reinforcement-learning-1/
开始写强化学习这个系列的文章,主要是为了记录学习过程中学习到的知识点,分享自己的理解。 强化学习简介强化学习(英语:Reinforcement learning,简称RL)是机器学习中的一个领域,强调如何基于环境而行动,以取得最大化的预期利益。其灵感来源于心理学中的行为主义理论,即有机体如何在环境给予的奖励或惩罚的刺激下,逐步形成对刺激的预期,产生能获得最大利益的习惯性行为。 强化学习涉及很多学
http://zicesun.com/2019/06/09/201906092129-spark-java-1/#more
用java实现简单的Spark例子。
http://zicesun.com/2019/03/27/1903271019-tencent-picdb/#more
之前都是使用新浪微博图床,但是最近这个图床一直在维护,其他的图床使用都比较麻烦,因此在这里我打算使用腾讯云的对象存储作为图床。
http://zicesun.com/2018/04/03/hello-world/
Welcome to Hexo! This is your very first post. Check documentation for more info. If you get any problems when using Hexo, you can find the answer in troubleshooting or you can ask me on GitHub. Quick
http://zicesun.com/2019/03/26/1903262337-hiv-introduction-1/#more
本文简单介绍hive的基本组件和简单的执行流程,是笔记本上的照片
http://zicesun.com/2018/06/10/20180610-reinforcement-learning-MDP/#more
上一篇文章强化学习——简介简单介绍了一下强化学习的相关概念。这篇博客将引入 马尔科夫决策过程(Markov Decision Processes, MDPs)对强化学习进行建模。这篇文章,将对马尔科夫决策过程以及Q-leaning进行介绍。 马尔科夫过程 定义: 若随机过程 $ \left \lbrace X_n, n \in T \right \rbrace$对于任意非负正整数$n \in T
http://zicesun.com/2019/04/02/201904021108-spark-scheduler/#more
大鲨鱼的博客《Spark Scheduler内部原理剖析》中,对spark scheduler的内部原理的分析还是很到位的。现在把这篇文章转载,详细请点击链接。http://sharkdtu.com/posts/spark-scheduler.html
http://zicesun.com/2019/03/27/201903271913-spark-learning-1/
之前看了一些Spark的相关内容,我觉得很有必要进行总结一下,不然都搭不起自己的知识框架。
http://zicesun.com/2019/04/11/201904110935-spark-learning-6/#more
这里,从源码的角度总结一下Spark RDD算子的用法。
http://zicesun.com/2019/04/05/201904052000-algorithm-overview/#more
在网上看到一个大佬整理的数据机构和算法系列,而且归纳的很好,附上链接,仅供参考。
http://zicesun.com/2019/06/10/201906100053-OS-Process-1/#more
进程在操作系统中是一个非常重要的概念,这篇文章将要介绍一下进程。
https://zicesun.com/2019/03/23/Spark%E2%80%94%E2%80%94RDD/#more
本文是转载守护之鲨关于Spark RDD的文章。
http://zicesun.com/2019/04/09/201904091637-spark-learning-4/#more
Spark的调度机制是Spark的核心基础,分成两个部分,一是集群资源调度,二是应用程序内部的作业调度。这篇文章简单介绍一下基本概念。
http://zicesun.com/2019/04/09/201904091552-spark-partitioner/#more
偶然看到一篇关于spark Partitioner的源码分析的好文,转载过来看一下。
http://zicesun.com/2019/03/28/201903281452-spark-learning-3/
Spark是一个基于分布式内存的大数据计算框架,RDD (Resilient Distributed Dataset)是Spark最重要的一个数据抽象。这篇文章记录了我对RDD的一些理解,有不足和错误的地方,请留言指正。
http://zicesun.com/2019/03/26/1903261608-git-upload-files/#more
使用git命令上传文件到github中
http://zicesun.com/2019/01/12/RNN-based-name-gender-prediction/#more
根据人的名字判断任务的性别是一个很有趣的工作.
http://zicesun.com/2019/07/07/201907071452-IO-model-1/#more
之前看了一点关于IO的模型,在看JAVA的时候,IO也是很重要的部分,其中最关键的NIO也是重中之重。在Java中,有三种IO模型,分别是阻塞IO(Block IO),非阻塞IO(NIO)和异步IO(AIO)。本篇笔记中的IO模型指的是操作系统中的IO模型,上面JAVA中的IO模型是只是Java API中的几种IO实现。
http://zicesun.com/2019/03/27/201903271019-tencent-picdb/
之前都是使用新浪微博图床,但是最近这个图床一直在维护,其他的图床使用都比较麻烦,因此在这里我打算使用腾讯云的对象存储作为图床。
http://zicesun.com/2019/03/28/1903281452-spark-learning-3/#more
Spark是一个基于分布式内存的大数据计算框架,RDD (Resilient Distributed Dataset)是Spark最重要的一个数据抽象。这篇文章记录了我对RDD的一些理解,有不足和错误的地方,请留言指正。
http://zicesun.com/2019/06/10/201906101705-multiThread-1/
本文主要介绍多线程的相关概念。
http://zicesun.com/2019/03/26/201903261608-git-upload-files/
使用git命令上传文件到github中
http://zicesun.com/2019/03/27/1903272048-spark-learning-2/#more
本文讲介绍如何搭建spark集群。
http://zicesun.com/2019/04/10/201904101128-spark-learning-5/#more
Spark的集群调度其实蛮简单的,这里总结了一些集群资源调度的基本原理。
http://zicesun.com/2019/06/28/201906281609-kafka-consumer/#more
Kafka消费者,生产者生产消息,并将消息发送到服务器中。而消费者则是根据topic订阅消息,从服务器上拉取消息进行处理。
http://zicesun.com/2019/06/28/201906281603-kafka-producer/#more
Kafka是分布式消息发布和订阅的消息系统。而生产者则对应的是消息的生产和发布。也就是说,生产者是消息的来源。生产者将消息发送搭kafka中。
https://zicesun.com/2019/01/12/hello-world/#more
Welcome to Hexo! This is your very first post. Check documentation for more info. If you get any problems when using Hexo, you can find the answer in troubleshooting or you can ask me on GitHub. Quick
https://zicesun.com/2019/01/11/pytorch-dataLoder-datasets/
在学习的过程中,遇到很多的问题.在训练模型的时候,需要对数据进行读取操作.本篇文章,介绍pytorch如何自定义数据dataset和dataloader.在pytorch中,提供了Dataset这个类,负责对数据进行抽象,一次调用只返回第一个同样本.而Dataloader提供了对一个对一个batch的数据操作,还有对数据进行shuffle等功能. DatasetDataset是一个抽象类,表示一个
http://zicesun.com/2019/03/27/201903272048-spark-learning-2/
本文讲介绍如何搭建spark集群。
http://zicesun.com/2019/03/26/201903262337-hiv-introduction-1/
本文简单介绍hive的基本组件和简单的执行流程,是笔记本上的照片。更为详细的内容参考文后的引用。
http://zicesun.com/2019/06/16/201906162114-kafka-1/#more
Kafka是一个消息队列。
http://zicesun.com/2018/06/03/reinforcement-learning-1/
开始写强化学习这个系列的文章,主要是为了记录学习过程中学习到的知识点,分享自己的理解。 强化学习简介强化学习(英语:Reinforcement learning,简称RL)是机器学习中的一个领域,强调如何基于环境而行动,以取得最大化的预期利益。其灵感来源于心理学中的行为主义理论,即有机体如何在环境给予的奖励或惩罚的刺激下,逐步形成对刺激的预期,产生能获得最大利益的习惯性行为。 强化学习涉及很多学
http://zicesun.com/2019/03/26/hello-world/#more
Welcome to Hexo! This is your very first post. Check documentation for more info. If you get any problems when using Hexo, you can find the answer in troubleshooting or you can ask me on GitHub. Quick
http://zicesun.com/2018/04/03/publish-first-blog/
搞了一下午和一晚上,博客终于弄好了,现在来测试一下。献上一首很喜欢的诗《致橡树》
http://zicesun.com/2018/04/03/201803272000-hello-world/#more
Welcome to Hexo! This is your very first post. Check documentation for more info. If you get any problems when using Hexo, you can find the answer in troubleshooting or you can ask me on GitHub. Quick
CS/reading
http://zicesun.com/2019/04/09/201904090108-spark-resource-links/#more
这列列举一些spark的学习资源和好的博客。
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.