Other articles

  1. covid-api - a free and open source API service for COVID-19 data

    Introduction

    In this period of COVID-19 emergency, many countries are publishing COVID related data that is being used by many existing projects and researchers.

    The main problem with these data is that they are being released in CSV format on some GitHub repository. While we fully appreciate the opennes of this format, unfortunataly it can introduce an additional work to be done (downloading the data, cleaning it, importing the data into a database, keeping it updated etc...) before someone can consume and analyse the data.

    covid-api

    covid-api project is a free and open source API service which automatically imports the data from various sources (at the moment we support the John Hopkins CSSE data source) and makes it available as a REST API.

    The service is still under development, but an initial version (with regularly updated data) is already available at https://api.covid19data.cloud.

    How to use the data

    To consume the API you don't need an account nor you need to authenticate in any way. You just need to request the right endpoint using the supported parameters.

    Here is an example for Python language:

    In [1]: import requests
    
    In [2]: response = requests.get('https://api.covid19data.cloud/v1/jh/daily-reports?last_update_from=2020-04-01&last_update_to=2020-04-03&country=Italy')
    
    In [3]: response.json()
    Out[3]:
    [{'id': 35343,
    'country_region': 'Italy',
    'province_state': None,
    'fips': None,
    'admin2': None,
    'last_update': '2020-04-01T21:58:34',
    'confirmed': 110574,
    'deaths': 13155,
    'recovered': 16847},
    {'id': 37895,
    'country_region': 'Italy',
    'province_state': None,
    'fips': None,
    'admin2': None,
    'last_update': '2020-04-02T23:25:14',
    'confirmed': 115242,
    'deaths': 13915,
    'recovered': 18278}]
    

    Further API documentation is available at https://api.covid19data.cloud/docs

    Next steps

    While we keep polishing the code and improving the existing data import procedure, we are planning to support additional data sources. The next one we are going to support is the Italian Protezione Civile.

    If you are aware of an additional data source that you would like to see covered, please let us know (creating a new Issue on GitHub) or send us a pull request.

    Contribute to the project

    If you are a Python developer and would like to contribute to the project, my advice is to first have a look at the main documentation available in the README.

    Then I suggest to have a look at the existing Issues and see where help is needed or in alternative you can open a new Issue or send a pull request with fixes and improvements.

    I also recommend to become familiar with our Code of Conduct before sending any contribution.

    Sponsors and Thanks

    I want to thank Heroku for accepting to sponsor the hosting of this service.

    I also want to thank all the volunteers involved in the project for their help and contributions.

    Disclaimer

    We are doing our best to keep the available data updated, clean (removing duplicates), and to provide a reliable service, but we are not in any way responsible for the accuracy of the data nor for the availability of the service itself. Please use it at your own risk.

    Abuse notice: we are currently not requiring any registration or authentication to use this service because we would like to keep it as simple as possible. Please do not abuse the service or you will force us to require a registration (subject to approval) to continue using it.

    read more

    comments

  2. Google is moving some EU citizens data to US

    What is going on

    As you may have heard already, because of brexit, Google is moving UK citizens data from the Northern Ireland data controller to the US one (Google LLC). Leaving the EU, UK citizens are not protected anymore by GDPR, and while this may be unfair, Google is legally allowed to do it.

    The problem

    Even if I'm an Italian citizen and I live in Italy, a few days ago I received this email from them:

    What's wrong with it?

    The point is that I'm an Italian citizen, living in Italy. I have nothing to do with UK (even if I lived there for a few years in the past, my account was created from Italy).

    Why do they mention "UK leaving EU" to me, if I don't live in UK?

    I tried to contact them multiple times on their @Google account on Twitter, but I got no reply at all. I tried to search online and it looks like I'm not alone, they are doing this to many other people: https://support.google.com/accounts/thread/29317992?hl=en&authuser=1

    Looking for help

    What should I do? Is this legally allowed?

    If there was an easy way to complain with them, I would have done it already, but I've tried to search on their website (even googling it... no pun intended) but I couldn't find a single contact form to report this issue and of course they are ignoring both Twitter and that forum I linked previously.

    Should I report them to the Privacy Authority? If yes, how?

    Full text of the email

    Here is the full text of the email I received:

    We’re improving our Terms of Service and making them easier for you to understand. 
    The changes will take effect on 31 March 2020, and they won’t impact the way that you use 
    Google services. And, because the United Kingdom (UK) is leaving the European Union (EU), 
    Google LLC will now be the service provider and the data controller responsible for your 
    information and for complying with applicable privacy laws for UK consumer users.
    
    For more details, we’ve provided a summary of the key changes and Frequently asked questions. 
    And the next time that you visit Google, you’ll have the chance to review and accept the new Terms. 
    At a glance, here’s what this update means for you:
    
    •   Improved readability: While our Terms remain a legal document, we’ve done our best to make them 
        easier to understand, including by adding links to useful information and providing definitions.
    •   Better communication: We’ve clearly explained when we’ll make changes to our services 
        (like adding or removing a feature) and when we’ll restrict or end a user’s access. 
        And we’ll do more to notify you when a change negatively impacts your experience on our services.
    •   Adding Google Chrome, Google Chrome OS and Google Drive to the Terms: Our improved Terms now 
        cover Google Chrome, Google Chrome OS and Google Drive, which also have service-specific terms 
        and policies to help you understand what’s unique to those services.
    •   Your service provider and data controller is now Google LLC: Because the UK is leaving the EU, 
        we’ve updated our Terms so that a United States-based company, Google LLC, is now your service 
        provider instead of Google Ireland Limited. Google LLC will also become the data controller 
        responsible for your information and complying with applicable privacy laws. 
        We’re making similar changes to the Terms of Service for YouTube, YouTube Paid Services and 
        Google Play. These changes to our Terms and privacy policy don’t affect your privacy settings 
        or the way that we treat your information (see the privacy policy for details). 
        As a reminder, you can always visit your Google Account to review your privacy settings and 
        manage how your data is used.
    
    If you’re the guardian of a child under the age required to manage their own Google Account and 
    you use Family Link to manage their use of Google services, please note that when you accept 
    our new Terms, you do so on their behalf as well, and you may want to discuss these changes with them.
    
    And of course, if you don’t agree to our new Terms and what we can expect from each other as you 
    use our services, you can find more information about your options in our Frequently asked questions.
    
    Thank you for using Google’s services.
    Your Google team
    
    read more

    comments

  3. How to deploy a static website to Github Pages using CircleCI

    Since I created my blog with a static pages generator, I've been using TravisCI to automate the pages build and deployment. My desire to learn something new (we are using CircleCI at work, but I never configured it from scratch) and the recent news about TravisCI acquisition and employees layoff, led me to think about moving to a different service.

    Github Pages

    Every account on Github can use a special repository to publish static pages. In my case, since I have github.com/andreagrandi, my special repository is named github.com/andreagrandi.github.io. Once I publish my pages there, they will be accessible from https://andreagrandi.github.io.

    You will need to use the master branch of the special repository directly and not the gh-pages branch which is available to each repository.

    CircleCI

    CircleCI is a very flexible and powerful continuous integration tool, which is also free for open source projects. As long as your static website is located on a public repository on Github, you won't have to pay anything to use it. In my case, the surce code of this website is available at https://github.com/andreagrandi/andreagrandi.it

    Configuration

    You can find the complete configuration at this address. The only value you won't find is GH_TOKEN. You need to generate this token on Github, at this address: https://github.com/settings/tokens. Give it a nice description like "CircleCI deployment token", select repo scope and finally click Generate token button. This token will be used to git push... your pages once they are built. Please remember to keep this token secret and not to publish it anywhere.

    In my configuration you may notice that I'm using Pelican static websites generator, but apart from a few changes, the structure of the configuration should be very similar even if you use Jekill, Hugo etc... it doesn't really matter how you generate the pages, the deployment phase will be the same.

    Deployment script

    You will notice that there is a complete bash script embedded in the CircleCI configuration. This script configures git, fetches the existing andreagrandi.github.io repository, and sync the built pages with the existing ones (this avoid creating a commit which contains all the pages so it will contain just the added content). Once the commit is made, the script will finally push the changes to the repository.

    Please note: regardless of CircleCI settings, the deployment will only happens if we are pushing (or merging a pull request) to master (if [ "${CIRCLE_BRANCH}" = "master" ]; then) and it will actually commit and push pages only if there is something new to commit (if git commit -m "CircleCI build $CIRCLE_BUILD_NUM pushed to Github Pages" ; then). For example if I'm just updating something in the CircleCI configuration, which doesn't change anything in the content, the pages won't be deployed again.

    Conclusion

    My first impression of CircleCI is that is faster than TravisCI and this means that I can publish my content more quickly. The possibility of using Docker containers as base image is really powerful and in more complex scenarios we can reproduce the building environment locally on our machine. If you have any advices about how to improve my build script, feel free to leave a comment here.

    read more

    comments

Page 1 / 51 »

social