How to install every R package your scripts require using Python

One of the annoying things many R users probably face is the annoyance of having to re-install all the R packages you use.

This can be necessary for a variety of reasons. You might buy a new computer or upgrade (as I did recently) to a new version of R, or you might want to dual boot your Windows computer with Linux.

Either way, you can be in a situation where you need to re-install all of your packages. This can be a pain.

So, I have written a Python script that does this for you. In essence it looks through your entire computer for R files and first finds where they have the words “require” or “library”. It then strips out the names of the packages required. Finally, it creates and then calls an R script which will check if these packages are installed and if they are not, it will install them.

I wrote this script because I had installed R 3.2 and all of my packages were wiped and I figured it would be easier to write this script than the annoyance of reinstalling them when I needed them. I am posting the code here in case anyone else needs it.

If people find it useful I might recode it in R.

Here is the code. Just copy and paste and save it as “InstallRPackages.py” and run from your home directory or wherever.


##############
import os

####### This is the root directory to start the search from. This may need modified ####
rootdir = '.'

#### Function to find the indices of all occurences of something in a string
def find_all(a_str, sub):
    start = 0
    while True:
        start = a_str.find(sub, start)
        if start == -1: return
        yield start
        start += len(sub) # use start += 1 to find overlapping matches
#### List of packages that need to be checked. This will be populated as it runs through the R code
packagecheck = list()

for subdir, dirs, files in os.walk(rootdir):
    for file in files:
		if ".R" in file:
			f = open(os.path.join(subdir, file))
			for line in f:
				if "library" in line or "require" in line:
					data = line.split(";")
					for i in data:
						if i.replace(" ", "").replace("require", "install.packages").replace("library", "install.packages").replace('(', '("').replace(')', '")').replace('""', '"').replace(")install",");install").replace("}", ")")[0:17] == "install.packages(":
							pack2print = i.replace(" ", "").replace("require", "install.packages").replace("library", "install.packages").replace('(', '("').replace(')', '")').replace('""', '"').replace(")install",");install").replace("}", ")").replace(")install",");install")
							if len(list(find_all(pack2print, ','))) > 0:
								pack2print = pack2print[0:list(find_all(pack2print, ','))[0]+1]
								pack2print = pack2print.replace(',',')')							
							if len(list(find_all(pack2print, '"'))) > 1:
								pack2print = pack2print[0:list(find_all(pack2print, '"'))[1]+2]
								### Now strip out the stuff and leave the package name								
								pack2print = pack2print.replace("install.packages(", "").replace(")","")
								### Add the package to the package list if it's not there already
								if pack2print not in packagecheck:
									packagecheck.append(pack2print)


text_file = open("InstallPackages.R", "w")
text_file.write('options(repos=structure(c(CRAN="http://cran.cnr.berkeley.edu/")))\n')

for pp in packagecheck:
	text_file.write("if(" + pp + "%in% installed.packages() != T)\n")
	text_file.write("{\n")
	text_file.write('print("Installing ' + pp.replace('"','') + '")\n')
	text_file.write("\t install.packages(" + pp + ", quiet = T)\n")
	text_file.write("}\n")

text_file.close()
import os
os.system("Rscript InstallPackages.R")
print "finished"

If the Green Party gets elected, will the last biologist to leave Britain turn off the lights!

Many scientists are more than a little skeptical of the Green Party. But, it is difficult to not welcome their announcement that they would raise science spending to 1% of GDP.

So, if elected, the Green Party would greatly raise spending on science. Sadly, they would also destroy vast areas of science at the same time, and essentially force many, if not most, biologists to shut up shop and find jobs in other countries. Continue reading

Jonathan Franzen on climate change in the New Yorker

The latest edition of the New Yorker has a pessimistic and rather misguided piece on climate change by Jonathan Franzen.

Here he is arguing that “alternative” energy may not be so desirable:

Even in the nations most threatened by flooding or drought, even in the countries most virtuously committed to alternative energy sources, no head of state has ever made a commitment to leaving any carbon in the ground. Without such a commitment, “alternative” merely means “additional”—postponement of human catastrophe, not prevention. The Earth as we now know it resembles a patient whose terminal cancer we can choose to treat either with disfiguring aggression or with palliation and sympathy. We can dam every river and blight every landscape with biofuel agriculture, solar farms, and wind turbines, to buy some extra years of moderated warming. Or we can settle for a shorter life of higher quality, protecting the areas where wild animals and plants are hanging on, at the cost of slightly hastening the human catastrophe. One advantage of the latter approach is that, if a miracle cure like fusion energy should come along, there might still be some intact ecosystems for it to save.

The analogy of the earth with a cancer patient here is nonsensical. The earth does not care that we put a wind farm in a wild place or a solar farm in a desert; it is humans that care. Sand and the rocks of mountains don’t have feelings.

One man’s blight is another man’s sublimity. Just think of those engineering marvels erected by the Victorians, many of them in so called wild places. The Ribblehead and Glenfinnan viaducts do not blight the landscape, they enhance it. But today, we appear to have become a species that apologises to the earth, and we seek engineering structures that are non-obtrusive. The rooftop solar panel is a symbol of a civilization that has lost its nerve.

No, we must enhance the landscape, not blight it. And why can this not be the case with wind farms? It certainly is the case for cooling towers, sublime structures now being demolished with the approval of short sighted politicians. A wild desolate landscape is the ideal place to stick a hundred or so wind turbines. The sheer scale is a sublime reminder of the limits of the human individual, and of the limitlessness of the human species.

Germany is the world’s biggest producer of lignite. Or is it?

Here is a claim I see frequently: Germany is the world’s biggest producer of brown coal (lignite). Useful rhetorical information for those critical of German energy policy, because lignite is just about the dirtiest fuel imaginable.

But is it true? According to the World Coal Association it is. According to them, Germany produces 183 Mt of brown coal. It also appears to be number one if you look at coal statistics reported by the International Energy Agency (IEA).

So, Germany must be the number one producer of lignite. The problem is that it is not. China is.

But, you won’t know this if you read IEA data. According to them, China produces and consumes no lignite whatsoever. However, a spreadsheet open in front of me tells me that China officially produced 265.66 Mt of lignite in 2009. And according to the Oxford Energy Institute, China now produces 370 Mt of lignite each year.

China, then, produces two times more lignite than Germany, and is the world number one.

Why the IEA and World Coal Association get this wrong is beyond me. The IEA splits China’s coal production into anthracite, coking, lignite, etc, and their split has little relationship with that given in China’s official data. And it’s not just lignite, the IEA claims China produces no anthracite, when it is close to half a billion tonnes in official data.

If someone knows why the IEA and World Coal Association don’t classify Chinese lignite as lignite in their statistics, please add a comment. Because I am perplexed.

A note on the perplexing situation

The IEA appears to officially define lignite as coal with calorific content less than 4,165 kcal/kg. However, Inner Mongolian produces 340 Mt of lignite each year, with an average calorific value of 3,500 kcal/kg, well below the level required to be classified as lignite by the IEA. So, there appears to be no reason for the IEA to classify none of China’s coal as lignite.

Nick Butler: India will soon inherit the unwanted title of the world’s largest single source of emissions

Nick Butler of the Financial Times is one of the more sensible commentators on energy out there. However, in his latest piece he has let the desire to construct a narrative get in the way of the facts.

Here is the troublesome paragraph:

China is now so central to the global energy market that whatever is done will send waves around the world. My bet is that we are about to see the Chinese government make a very serious move to limit coal use – with peak coal coming much sooner than the usual predictions suggest. Any advance in the technology of renewables anywhere in the world will be seized upon and applied. Gas imports will continue to grow. In each case global markets and prices will be affected. The policy will put great pressure on India which will soon inherit the unwanted title of the world’s largest single source of emissions.

Can you spot the implausible assertion?

Yes, it’s the claim that India will soon overtake China as the world’s number one emitter. Continue reading

Naomi Klein on promoting “sustainable” lifestyles

“The added irony is that many of the people being sacrificed for the carbon market are living some of the most sustainable, low-carbon lifestyles on the planet. They have strong reciprocal relationships with nature, drawing on local ecosystems on a small scale while caring for and regenerating the land so it continues to provide for them and their descendants. An environmental movement committed to real climate solutions would be looking for ways to support these ways of life – not severing deep traditions of stewardship and pushing more people to become rootless urban consumers”.

This is how Naomi Klein describes Amazonian peasants in her book This Change Everything.

The book is bad enough by this point (page 223), but once Klein put the above paragraph down on paper the book could no longer be redeemed. Irregardless of the delusions of Klein, likely fed by the greater delusions of the likes of Vandana Shiva, most peasants do not wish to remain peasants. Quite the contrary. Today we are seeing the greatest migration in human history, from the village to the city. Those living sustainable lives are choosing rootlessness.

Time to declare: Vote Ukip!

There is nothing more British than fish and chips. Disagree? Well, read this piece by Jay Rayner, explaining how fish and chips have been threatened by mass immigration throughout British history.

There is also nothing more British than beer. And no one makes better beer than the British. If you have ever been foolish to go on holiday to Belgium, you will know what I am talking about.

I also don’t like to read books, listen to music or watch TV.

Which is all rather fortunate. Because, up until today I was going to hold my nose and vote Labour. And then I saw this.

ukip

What’s not to like? I will now be voting Ukip.

Helen Czerski on “balanced” climate change coverage

Twitter can be a rather tiresome place, and it can also make previously interesting people tiresome. The desire for uninformed outrage is an unwelcome one.

There was another example of this this week, when a climate scientist (un-named) was complaining about a supposedly biased BBC documentary (un-watched by him).

In the middle of this argument came a tweet from Helen Czerski – a participant in the documentary – which is worth noting.

There you go, a simple statement of what a balanced media report on the science of climate change would actually require: no climate change “skeptics”.

Lies, damn lies, and energy statistics

Here is a simple and incredibly effective way to make it seem that a power plant will generate much more electricity than it actually will: talk purely about the amount of electricity it will supply to homes.

It’s a simple trick. It goes something like this: “Nuclear power plant to provide enough electricity for 5 million homes”; “Wind farms provide enough electricity for all of Scotland’s houses”. I’m sure you’ve seen it. Continue reading

Britain’s GHG emissions are down 8.4%. And more than half of the reductions were due to the weather

Tracking year to year changes in greenhouse gas emissions can be problematic. It is incredibly easy to read too much into what happens in a single year. And this can go in two directions.

Consider coal in China. Last year, China’s coal use declined (officially that is, but we can’t trust China’s coal statistics that much). If you are naive, you would believe that China has reached peak coal. This view is somewhat delusional given that China has over 100 gigawatts of coal power plants under construction.

So, what are we to make of the fact that Britain’s GHG emissions declined by 8.4% last year? Continue reading