ISBN：9787115448088
資深的資訊工程師常有一種特別的煩惱，如何在眾多實作技術中，挑選出最恰當的工具來執行軟體專案。從中牽扯的包括組織資源、同儕能力、工具成熟度、操作便易性，這項任務恰好沒有一體適用的標準答案，如同撰寫程式時「變數命名」般，十足仰賴負責人的主觀決擇。
大數據專案開發也會遇到一樣的問題，在今日開源軟體、雲端平台遍地開花的時代，要收集、分析與運用企業本身的巨量資料，可以從最精典的Map/Reduce框架到高效版本的Spark平台。不論企業最後挑選的大數據解決方案為何，勢必需要熟悉該平台的工程師運籌帷幄。
由於大數據開發有別於傳統的關聯式資料庫，因此對於工程團隊來說，大數據技術變成一個得跨越過去的門檻。而Apache Hive正是主張能一勞永逸解決這個技術「痛點」的開源工具。它無縫地運行在Hadoop平台之上，充分利用HDFS的分散式檔案系統以及Yarn平行處理框架，並在自身的對外介面套上一層相容SQL(HQL)的外衣，讓各項軟體工具、運用程式甚至是開發人員，都能用熟練的SQL語法，遨遊於巨量資料的國度中。
對我而言，由於已經熟練Map/Reduce與Spark等技術，可以隨心所欲地撰寫出大數據程式碼。其實沒必要疊床架屋的再添一層虛擬資料庫，用SQL語法來執行巨量運算。但考量到企業整體運用情境，想擴增大數據的應用面以及簡化同儕的數據使用門檻，就值得投入心力建置一個獨立運作的Hive環境。
市面上Hive的專書不多，而且絕大多數是四、五年前所出版。在決定再次架設Hive服務的當下，理所當然地挑選2020年出版的「大數據Hive離線計算開發實戰」。內文從最基礎的概念說起，再由簡入深地提及到分區、自定義函式等進階的功能，作者將Hive定位為能處理大數據的資料倉儲解決方案，全書也呼應這個想法具體地呈現如何實踐。閱後不免心有所思，在這麼強大便利的開源方案的競爭下，未來商用資料倉儲廠商肯定不好擴展市場。
作者用最後三章舉出大數據的常見運用情境，分別是推薦系統、網站流量分析以及商業數據分析。題目夠大但內容卻僅是入門級的基本範例。特別是推薦系統一章中，是用Mahout來執行協同過瀘演算法來做範例。先不論將推薦系統，用單一個協同過瀘法來涵蓋，是否會過於簡化問題以致誤導讀者。書中介紹的Mahout版本(運行在Hadoop 2.x上)，已經是太舊的版本，令人懷疑這本書的出版年份是否有誤植？
總得來說本書可以讓讀者快速領略Hive在大數據平台中所能扮演的角色，免除耗費心力瀏覽散落各處的官網文件，最後還落得一頭霧水。如果動心起念想要著手大數據運算，Hive和這本「大數據Hive離線計算開發實戰」算是不錯的起點。
Senior software engineers often face a particular kind of dilemma: how to choose the most appropriate tools from a vast array of implementation technologies for a software project. The decision involves considerations such as organizational resources, team capabilities, tool maturity, and ease of use. Unfortunately, there is no universally applicable answer to this challenge. Much like “naming variables” in programming, the final decision relies heavily on the subjective judgment of the person in charge.
Big data projects encounter the same issue. In today’s era of flourishing open-source software and cloud platforms, enterprises seeking to collect, analyze, and leverage massive volumes of data can choose from everything from the classic MapReduce framework to high-performance platforms such as Spark. Regardless of which big data solution an organization ultimately adopts, it will inevitably require engineers who are proficient in that platform to guide and manage the project.
Because big data development differs significantly from traditional relational databases, mastering big data technologies has become a hurdle engineering teams must overcome. Apache Hive is an open-source tool that claims to solve this technical “pain point” once and for all. It operates seamlessly on top of the Hadoop ecosystem, fully leveraging HDFS as a distributed file system and YARN as a parallel processing framework. On top of that, Hive provides an SQL-compatible interface (HQL), allowing software tools, applications, and even developers themselves to navigate the world of big data using the familiar syntax of SQL.
From my perspective, since I am already proficient in technologies such as MapReduce and Spark, I can freely develop big data applications directly. Strictly speaking, there is no real need to add another layer of abstraction in the form of a virtual database and execute large-scale computations through SQL. However, when considering enterprise-wide adoption, expanding the accessibility of big data applications and lowering the barrier for colleagues to work with data make it worthwhile to invest in building an independently operated Hive environment.
There are not many books on Hive available on the market, and most of them were published four or five years ago. Therefore, when I decided to set up a Hive service once again, it was only natural to choose the 2020 publication Big Data Hive Offline Computing Development in Practice. The book begins with the most fundamental concepts and gradually progresses toward advanced features such as partitioning and user-defined functions. The author positions Hive as a data warehouse solution capable of handling big data, and the entire book consistently demonstrates how to implement that vision in practice. After finishing the book, one cannot help but reflect that under the competition of such powerful and convenient open-source solutions, commercial data warehouse vendors will likely find it increasingly difficult to expand their markets in the future.
In the final three chapters, the author presents several common big data application scenarios, including recommendation systems, website traffic analysis, and business data analytics. Although the topics themselves are broad, the actual content remains at an introductory level with only basic examples. In particular, the chapter on recommendation systems uses Mahout to demonstrate collaborative filtering algorithms. Setting aside the question of whether using only a single collaborative filtering approach oversimplifies recommendation systems and potentially misleads readers, the version of Mahout introduced in the book (running on Hadoop 2.x) is already severely outdated. This raises doubts as to whether the publication year of the book may have been printed incorrectly.
Overall, this book allows readers to quickly grasp the role Hive can play within a big data platform, sparing them the effort of piecing together scattered official documentation only to end up more confused. For anyone interested in getting started with big data computing, both Hive and Big Data Hive Offline Computing Development in Practice serve as solid starting points.
