A Study of Big Data Challenges and Opportunities

Abstract
Authors
Keywords
Conclusion
References

The Big Data” as a term has been among the biggest trends of the last three years, leading to an upsurge of research, as well as industry and government applications. As per the user demand and growth trends of large free data, the storage solutions are now becoming challengeable to protect, store and retrieve data. The days are not so far when the storage companies and organizations will start saying ‘no’ to store your valuable data or they will start charging a huge amount for its storage and protection. The flood of big data will lead to the zettabyte per year range with in a little time period. The major attributes of big data to be emphasized are volume, velocity, variety and veracity and it always looks like that the storage issue will be resolved in near future but it is a long duration challenge. In this paper we have analyzed the growth trend of big data and its future projection. We have also focused on the impact of the unstructured data on various concerns and we have also suggested some possible remedies to streamline big data.

Published In : IJCAT Journal Volume 3, Issue 3

Date of Publication : April 2016

Pages : 250-254

Figures :02

Tables : --

Publication Link :A Study of Big Data Challenges and Opportunities

Amarbir Singh : is pursuing Ph. D from Punjab Technical University. He has done Master of Computer Applications from Guru Nanak Dev University, Amritsar in 2006 and has published more than ten research papers in various international journals and conferences.

Sarabjit Singh : is currently working as assistant professor in Guru nanak Dev University College Verka, Amritsar. He has done Master of Computer Applications from Guru Nanak Dev University, Amritsar in 2005 and has published more than 8 research papers in various international journals and conferences.

Structured Data, Unstructured Data, Veracity, Hadoop.

In this paper, we have analyzed the basic concept, characteristics & need of Big Data. The impact of social sites, private data and global warming factors are discussed in this paper. Our analysis illustrate that the growth trend of big data is because of unwanted and unstructured data. The regular monitoring and regular deletion of unwanted and duplicated data are few possible solutions to control the growth rate of big data. Security, storage, searching and environmental changes are the biggest issues related with Big Data and they must be handled carefully in order extract maximum benefit from the big data.

[1] Kaisler, S., W. Money, and S. J. Cohen. 2012. “A Decision Framework for Cloud Computing”, 45th Hawaii International Conference on System Sciences, Grand Wailea, Maui, HI, Jan 4-7, 2012. [2] Shiwen Mao · Yunhao Liu "Big Data: A Survey" Min Chen · Published online: 22 January 2014 © Springer Science+Business Media New York 2014. [3] World's data will grow by 50X in next decade, http://www.computerworld.com/s/article/9217988/Wo rld-s-data-will-grow-by-50X-in- next-decade-IDCstudy- predicts. [4] IDC Releases First Worldwide Big Data Technology and Services Market Forecast, Shows Big Data as the Next Essential Capability and a Foundation for the Intelligent Economy http://outsourcing.ulitzer.com/node/2195534. [5] J. A. Hanley and B. J. McNeil, “A method of comparing the areas under receiver operating characteristic curves derived from the same cases,” Radiology, vol. 148, no. 3, pp. 839–843, 1983. [6] W. Gao, R. Jin, S. Zhu, and Z.-H. Zhou, “One-pass AUC optimization,” in Proceedings of the 30th International Conference on Machine Learning, Atlanta, GA, 2013, pp. 906–914. [7] Y. Bengio, “Learning deep architectures for AI,” Foundations and Trends in Machine Learning, vol. 2, no. 1, pp. 1–127, 2009. [8] P. Baldi and P. J. Sadowski, “Understanding dropout,”, in Advances in Neural Information Processing Systems 26. Cambridge, MA: MIT Press, 2013, pp. 2814–2822. [9] W. Gao and Z.-H. Zhou, “Dropout Rademacher complexity of deep neural networks”, CORR abs/1402.3811, 2014. [10] S. Wager, S. Wang, and P. Liang, “Dropout training as adaptive regularization,” Advances in Neural Information Processing Systems 26. Cambridge, MA: MIT Press, 2013, pp. 351–359. [11] T. G. Dietterich, “Approximate statistical tests for comparing supervised classification learning algorithms,” Neural Computation, vol. 10, no. 7, pp. 1895 – 1923, 1998. [12] Z.-H. Zhou, “Rule extraction: Using neural networks or for neural networks?”, Journal of Computer Science and Technology, vol. 19, no. 2, 2004, pp. 249–253. [13] M. Li, W. Wang, and Z.-H. Zhou, “Exploiting remote learners in internet environment with agents,” Science China: Information Sciences, vol. 53, no. 1, 2010, pp. 47–76. [14] M. Mohri and A. Rostamizadeh, “Stability bounds for stationary -mixing and -mixing processes,” Journal of Machine Learning Research, vol. 11, pp. 789–814, 2010. [15] D. Zhou, J. C. Platt, S. Basu, and Y. Mao, “Learning from the wisdom of crowds by minimax entropy,” in Advances in Neural Information Processing Systems 25, P. Bartlett, F. C. N. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, Eds. Cambridge, MA: MIT Press, 2012. [16] N. Li, I. W. Tsang, and Z.-H. Zhou, “Efficient optimization of performance measures by classifier adaptation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 6, 2013,pp. 1370–1382.